Sep 18 17:47:46.675151 ip-10-0-130-66 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 18 17:47:46.675165 ip-10-0-130-66 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 18 17:47:46.675176 ip-10-0-130-66 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 18 17:47:46.675388 ip-10-0-130-66 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 18 17:47:56.924455 ip-10-0-130-66 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 18 17:47:56.924471 ip-10-0-130-66 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 9fc5d95806d54b3498dba91e2e0488e0 --
Sep 18 17:49:54.519478 ip-10-0-130-66 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 17:49:57.863051 ip-10-0-130-66 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 17:49:57.863051 ip-10-0-130-66 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 17:49:57.863051 ip-10-0-130-66 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 17:49:57.863051 ip-10-0-130-66 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 17:49:57.863051 ip-10-0-130-66 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 17:49:57.863872 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.863132    2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 17:49:57.867300 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867282    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 17:49:57.867300 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867299    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 17:49:57.867300 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867302    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867306    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867309    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867313    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867316    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867318    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867321    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867323    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867326    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867328    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867331    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867334    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867336    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867339    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867341    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867344    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867349    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867353    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867357    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867359    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 17:49:57.867400 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867362    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867365    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867368    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867370    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867373    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867376    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867378    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867381    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867383    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867386    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867389    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867391    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867394    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867397    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867402    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867406    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867410    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867413    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867416    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 17:49:57.867884 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867419    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867422    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867425    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867429    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867432    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867436    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867440    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867443    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867446    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867449    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867452    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867454    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867458    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867461    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867463    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867467    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867470    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867472    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867474    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 17:49:57.868396 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867477    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867480    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867482    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867485    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867487    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867490    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867492    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867496    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867499    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867502    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867504    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867507    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867509    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867511    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867514    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867517    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867520    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867522    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867525    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867527    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 17:49:57.868855 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867530    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867533    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867535    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867538    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867540    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867543    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867545    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867547    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867550    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867552    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867555    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867557    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867560    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867562    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867565    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867567    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867570    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867573    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.867575    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868295    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 17:49:57.869352 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868303    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868306    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868309    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868312    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868315    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868317    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868320    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868323    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868326    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868328    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868331    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868334    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868336    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868340    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868343    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868346    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868349    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868351    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868354    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868357    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 17:49:57.869837 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868359    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868362    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868364    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868367    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868369    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868372    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868375    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868377    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868380    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868382    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868386    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868389    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868392    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868395    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868397    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868400    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868403    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868406    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868409    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 17:49:57.870369 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868412    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868414    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868417    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868420    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868423    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868425    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868427    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868430    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868433    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868435    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868437    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868440    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868442    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868445    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868448    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868450    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868453    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868455    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868459    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868461    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 17:49:57.870882 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868464    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868466    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868469    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868472    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868475    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868478    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868480    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868483    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868485    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868488    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868491    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868493    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868496    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868498    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868502    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868506    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868509    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868512    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868516    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 17:49:57.871399 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868520    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868524    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868527    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868529    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868532    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868535    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868538    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868540    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868543    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868546    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868549    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868551    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868554    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868556    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868559    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868562    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868564    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868567    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868570    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868573    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 17:49:57.871865 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868952    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868957    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868960    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868963    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868966    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868969    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868971    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868974    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868979    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868982    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868985    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868988    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868990    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868993    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868995    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.868998    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869001    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869005    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869008    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 17:49:57.872368 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869011    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869014    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869017    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869019    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869022    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869024    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869027    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869030    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869032    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869035    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869038    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869041    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869043    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869046    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869051    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869054    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869057    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869060    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869063    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 17:49:57.872841 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869066    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869068    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869071    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869074    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869076    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869079    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869081    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869084    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869086    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869088    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869091    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869094    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869096    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869099    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869101    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869104    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869107    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869109    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869111    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869115    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 17:49:57.873351 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869117    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869120    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869123    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869125    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869128    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869130    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869133    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869135    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869138    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869140    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869143    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869146    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869148    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869150    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869153    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869156    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869160    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869163    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869166    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 17:49:57.873844 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869169    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869172    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869188    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869192    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869195    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869198    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869201    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869206    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869210    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869212    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869215    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869218    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869220    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869223    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869225    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869228    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869231    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869233    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869236    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869239    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 17:49:57.874342 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869241    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869244    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869312    2568 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869343    2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869351    2568 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869355    2568 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869360    2568 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869363    2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869368    2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869372    2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869376    2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869379    2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869383    2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869386    2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869390    2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869393    2568 flags.go:64] FLAG: --cgroup-root=""
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869396    2568 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869399    2568 flags.go:64] FLAG: --client-ca-file=""
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869402    2568 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869406    2568 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869410    2568 flags.go:64] FLAG: --cluster-domain=""
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869413    2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869417    2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869420    2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 17:49:57.874821 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869423    2568 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869427    2568 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869430    2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869435    2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869439    2568 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869442    2568 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869445    2568 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869448    2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869451    2568 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869454    2568 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869458    2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869461    2568 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869464    2568 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869467    2568 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869470    2568 flags.go:64] FLAG: --enable-server="true"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869473    2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869478    2568 flags.go:64] FLAG: --event-burst="100"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869481    2568 flags.go:64] FLAG: --event-qps="50"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869484    2568 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869488    2568 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869491    2568 flags.go:64] FLAG: --eviction-hard=""
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869495    2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869498    2568 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869501    2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869504    2568 flags.go:64] FLAG: --eviction-soft=""
Sep 18 17:49:57.875419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869507    2568 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869510    2568 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869514    2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869517    2568 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869520    2568 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869523    2568 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869525    2568 flags.go:64] FLAG: --feature-gates=""
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869529    2568 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869532    2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869535    2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869539    2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869543    2568 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869546    2568 flags.go:64] FLAG: --help="false"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869549    2568 flags.go:64] FLAG: --hostname-override="ip-10-0-130-66.ec2.internal"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869552    2568 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869555    2568 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869558    2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869562    2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869565    2568 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869568    2568 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869571    2568 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869574    2568 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869577    2568 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 17:49:57.876026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869580    2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869583    2568 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869586    2568 flags.go:64] FLAG: --kube-reserved=""
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869589    2568 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869593    2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869596    2568 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869599    2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869602    2568 flags.go:64] FLAG: --lock-file=""
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869605    2568 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869608    2568 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869611    2568 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869618    2568 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869621    2568 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869623    2568 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869626    2568 flags.go:64] FLAG: --logging-format="text"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869629    2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869633    2568 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869635    2568 flags.go:64] FLAG: --manifest-url=""
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869638    2568 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869644    2568 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869647    2568 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869651    2568 flags.go:64] FLAG: --max-pods="110"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869654    2568 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869657    2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869660    2568 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 17:49:57.876619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869663    2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869666    2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869669    2568 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869672    2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869679    2568 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869682    2568 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869685    2568 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869688    2568 flags.go:64] FLAG: --pod-cidr=""
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869691    2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869697    2568 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869700    2568 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869703    2568 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869707    2568 flags.go:64] FLAG: --port="10250"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869710    2568 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869713    2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-037b73183558710fa"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869717    2568 flags.go:64] FLAG: --qos-reserved=""
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869720    2568 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869723    2568 flags.go:64] FLAG: --register-node="true"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869727    2568 flags.go:64] FLAG: --register-with-taints=""
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869735    2568 flags.go:64] FLAG: --registry-burst="10"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869738    2568 flags.go:64] FLAG: --registry-qps="5"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869741    2568 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869744    2568 flags.go:64] FLAG: --reserved-memory=""
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869748    2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869751    2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 17:49:57.877293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869754    2568 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869757    2568 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869763    2568 flags.go:64] FLAG: --runonce="false"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869766    2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869769    2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869772    2568 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869776    2568 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869779    2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869782    2568 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869785    2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869788    2568 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869791    2568 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869794    2568 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869797    2568 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869799    2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869803    2568 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869806    2568 flags.go:64] FLAG: --system-cgroups=""
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869809    2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869814    2568 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869817    2568 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869821    2568 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869825    2568 flags.go:64] FLAG: --tls-min-version=""
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869828    2568 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869831    2568 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869834    2568 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 17:49:57.877893 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869839    2568 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869842    2568 flags.go:64] FLAG: --v="2"
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869846    2568 flags.go:64] FLAG: --version="false"
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869850    2568 flags.go:64] FLAG: --vmodule=""
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869854    2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.869857    2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869973    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869976    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869979    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869983    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869986    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869989    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869992    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869995    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.869997    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870000    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870003    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870005    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870009    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870011    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870014    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 17:49:57.878534 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870016    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870019    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870022    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870025    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870027    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870030    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870033    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870036    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870039    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870042    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870044    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870049    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870051    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870054    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870057    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870059    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870062    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870065    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870067    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870070    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 17:49:57.879063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870074    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870077    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870079    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870082    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870085    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870087    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870090    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870093    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870095    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870098    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870101    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870104    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870107    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870109    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870113    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870115    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870118    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870121    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870124    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870126    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 17:49:57.879566 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870130    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870134    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870138    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870143    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870146    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870148    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870151    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870161    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870164    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870167    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870170    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870186    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870192    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870196    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870202    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870206    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870209    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870212    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870215    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 17:49:57.880053 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870218    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870221    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870223    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870226    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870229    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870232    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870235    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870237    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870242    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870245    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870248    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870252    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870255    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870257    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870261    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870263    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870268    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870270    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870273    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870276    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 17:49:57.880527 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870278    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 17:49:57.881022 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870281    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 17:49:57.881022 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870284    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 17:49:57.881022 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870286    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 17:49:57.881022 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.870289    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 17:49:57.881022 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.870299    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 17:49:57.881022 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.877608    2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 17:49:57.881022 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.877627    2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 17:49:57.881022 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.877648    2568 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 17:49:57.881022 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.877657    2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 17:49:57.881022 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877722    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 17:49:57.881022 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877727    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 17:49:57.881022 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877731    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 17:49:57.881022 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877735    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 17:49:57.881022 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877739    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877742    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877746    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877749    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877752    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877755    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877758    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877761    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877764    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877767    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877770    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877773    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877776    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877779    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877782    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877785    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877788    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877791    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877794    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877796    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877799    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 17:49:57.881383 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877802    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877805    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877808    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877811    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877822    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877825    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877828    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877831    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877834    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877836    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877839    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877841    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877844    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877847    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877849    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877852    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877854    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877857    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877859    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877862    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 17:49:57.881894 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877865    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877867    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877870    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877872    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877875    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877877    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877880    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877882    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877885    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877887    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877890    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877893    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877895    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877898    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877901    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877903    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877907    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877915    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877918    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 17:49:57.882843 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877921    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877924    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877926    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877929    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877931    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877934    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877936    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877939    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877941    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877944    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877947    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877949    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877952    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877954    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877958    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877962    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877965    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877968    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877971    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 17:49:57.883512 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877975    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 17:49:57.884019 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877978    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 17:49:57.884019 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877980    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 17:49:57.884019 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877983    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 17:49:57.884019 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877985    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 17:49:57.884019 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877988    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 17:49:57.884019 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877991    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 17:49:57.884019 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877993    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 17:49:57.884019 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877996    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 17:49:57.884019 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.877998    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 17:49:57.884019 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878002    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 17:49:57.884019 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878004    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 17:49:57.884019 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878014    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 17:49:57.884019 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878017    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 17:49:57.884019 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878019    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 17:49:57.884019 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878022    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 17:49:57.884407 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.878027    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 17:49:57.884407 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878205    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 17:49:57.884407 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878211    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 17:49:57.884407 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878214    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 17:49:57.884407 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878217    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 17:49:57.884407 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878220    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 17:49:57.884407 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878223    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 17:49:57.884407 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878226    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 17:49:57.884407 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878228    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 17:49:57.884407 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878231    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 17:49:57.884407 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878233    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 17:49:57.884407 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878236    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 17:49:57.884407 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878239    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 17:49:57.884407 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878243    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878247    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878250    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878255    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878259    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878267    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878270    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878272    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878275    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878277    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878280    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878282    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878285    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878288    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878290    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878293    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878301    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878304    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878306    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 17:49:57.884766 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878309    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878312    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878314    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878317    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878319    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878322    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878324    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878327    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878331    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878334    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878337    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878340    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878343    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878346    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878349    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878352    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878355    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878357    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878360    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 17:49:57.885277 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878362    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878365    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878368    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878370    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878373    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878375    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878378    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878381    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878383    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878386    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878388    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878397    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878399    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878402    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878404    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878407    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878410    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878412    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878415    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878417    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878420    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 17:49:57.885751 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878422    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878425    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878427    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878430    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878432    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878435    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878437    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878440    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878443    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878445    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878448    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878451    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878453    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878456    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878459    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878461    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878464    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878466    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878469    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 17:49:57.886359 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878472    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 17:49:57.886947 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878474    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 17:49:57.886947 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878477    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 17:49:57.886947 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878479    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 17:49:57.886947 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878488    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 17:49:57.886947 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878491    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 17:49:57.886947 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878494    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 17:49:57.886947 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878496    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 17:49:57.886947 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:57.878499    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 17:49:57.886947 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.878504    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 17:49:57.886947 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.878629    2568 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 17:49:57.886947 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.880610    2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 17:49:57.886947 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.881351    2568 server.go:1019] "Starting client certificate rotation"
Sep 18 17:49:57.886947 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.881445    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 17:49:57.886947 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.881487    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 17:49:57.887357 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.885540    2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 17:49:57.887357 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.886703    2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 17:49:57.891550 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.891534    2568 log.go:25] "Validated CRI v1 runtime API"
Sep 18 17:49:57.893063 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.893047    2568 log.go:25] "Validated CRI v1 image API"
Sep 18 17:49:57.895127 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.895103    2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 17:49:57.897156 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.897137    2568 fs.go:135] Filesystem UUIDs: map[339ddcac-f70e-4602-97a8-ce9705800058:/dev/nvme0n1p3 5492a81c-9ad0-402a-bdcb-5208810494ed:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 18 17:49:57.897242 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.897154    2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 17:49:57.901912 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.901789    2568 manager.go:216] Machine: {Timestamp:2026-09-18 17:49:57.901299473 +0000 UTC m=+1.979103363 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100089 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec273522dd2cf73dd4a24fe66c2a7fb2 SystemUUID:ec273522-dd2c-f73d-d4a2-4fe66c2a7fb2 BootID:9fc5d958-06d5-4b34-98db-a91e2e0488e0 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c7:22:e5:4d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c7:22:e5:4d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:2d:c7:3e:c7:99 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 17:49:57.901912 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.901900    2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 17:49:57.902045 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.902002    2568 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 17:49:57.903344 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.903316    2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 17:49:57.903514 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.903345    2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-66.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 17:49:57.904503 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.904490    2568 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 17:49:57.904573 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.904511    2568 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 17:49:57.904573 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.904523    2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 17:49:57.904573 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.904538    2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 17:49:57.904707 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.904629    2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 17:49:57.908087 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.908072    2568 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 17:49:57.908235 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.908223    2568 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 17:49:57.908306 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.908271    2568 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 17:49:57.908306 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.908292    2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 17:49:57.908393 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.908307    2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 17:49:57.908393 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.908320    2568 kubelet.go:398] "Adding apiserver pod source"
Sep 18 17:49:57.908393 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.908332    2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 17:49:57.908707 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.908694    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 17:49:57.908797 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.908717    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 17:49:57.912942 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.912921    2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 17:49:57.915234 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.915218    2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 17:49:57.915308 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.915252    2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 17:49:57.916208 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.916197    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 17:49:57.916259 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.916213    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 17:49:57.916259 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.916219    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 17:49:57.916259 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.916224    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 17:49:57.916259 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.916253    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 17:49:57.916259 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.916259    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 17:49:57.916420 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.916274    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 17:49:57.916420 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.916280    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 17:49:57.916420 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.916287    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 17:49:57.916420 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.916292    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 17:49:57.916420 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.916301    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 17:49:57.916420 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.916306    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 17:49:57.916420 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.916332    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 17:49:57.916420 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.916338    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 17:49:57.917545 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:57.917521    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-66.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 17:49:57.917639 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:57.917617    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 17:49:57.919029 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.919010    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 17:49:57.920262 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.920249    2568 server.go:1268] "Started kubelet"
Sep 18 17:49:57.920390 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.920341    2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 17:49:57.920485 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.920411    2568 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 17:49:57.920695 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.920668    2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 17:49:57.921047 ip-10-0-130-66 systemd[1]: Started Kubernetes Kubelet.
Sep 18 17:49:57.921849 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.921682    2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 17:49:57.921914 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.921901    2568 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 17:49:57.925064 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:57.925041    2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 17:49:57.926831 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.926812    2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 17:49:57.926831 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.926826    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 17:49:57.926949 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:57.926913    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Sep 18 17:49:57.927530 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.927457    2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 17:49:57.927530 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.927464    2568 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 17:49:57.927530 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.927478    2568 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 17:49:57.927706 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.927671    2568 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 17:49:57.927706 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.927689    2568 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 17:49:57.927830 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.927814    2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-66.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 17:49:57.927886 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.927871    2568 factory.go:55] Registering systemd factory
Sep 18 17:49:57.927928 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.927889    2568 factory.go:223] Registration of the systemd container factory successfully
Sep 18 17:49:57.928123 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.928107    2568 factory.go:153] Registering CRI-O factory
Sep 18 17:49:57.928213 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.928125    2568 factory.go:223] Registration of the crio container factory successfully
Sep 18 17:49:57.928213 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.928206    2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 17:49:57.928296 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.928229    2568 factory.go:103] Registering Raw factory
Sep 18 17:49:57.928296 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.928245    2568 manager.go:1196] Started watching for new ooms in manager
Sep 18 17:49:57.928535 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.928525    2568 manager.go:349] Starting recovery of all containers
Sep 18 17:49:57.949261 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.949222    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 17:49:57.951635 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:57.951601    2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-66.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 18 17:49:57.951853 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:57.951826    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 18 17:49:57.951943 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.951881    2568 manager.go:354] Recovery completed
Sep 18 17:49:57.956241 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:57.955344    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-66.ec2.internal.18d67b98c070eabf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-66.ec2.internal,UID:ip-10-0-130-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-66.ec2.internal,},FirstTimestamp:2026-09-18 17:49:57.920221887 +0000 UTC m=+1.998025777,LastTimestamp:2026-09-18 17:49:57.920221887 +0000 UTC m=+1.998025777,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-66.ec2.internal,}"
Sep 18 17:49:57.957372 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.957360    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 17:49:57.958989 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.958970    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 17:49:57.959042 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.959007    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 17:49:57.959042 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.959019    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Sep 18 17:49:57.959678 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.959659    2568 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 17:49:57.959678 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.959673    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 17:49:57.959777 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.959675    2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 17:49:57.959777 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.959743    2568 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 17:49:57.960408 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.960393    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 17:49:57.960469 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.960420    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 17:49:57.960469 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.960430    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Sep 18 17:49:57.961784 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.961772    2568 policy_none.go:49] "None policy: Start"
Sep 18 17:49:57.961819 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.961786    2568 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 17:49:57.961819 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.961797    2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 17:49:57.963726 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.963715    2568 policy_none.go:47] "Start"
Sep 18 17:49:57.969963 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:57.969895    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-66.ec2.internal.18d67b98c2c07ac4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-66.ec2.internal,UID:ip-10-0-130-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-66.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-66.ec2.internal,},FirstTimestamp:2026-09-18 17:49:57.958990532 +0000 UTC m=+2.036794421,LastTimestamp:2026-09-18 17:49:57.958990532 +0000 UTC m=+2.036794421,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-66.ec2.internal,}"
Sep 18 17:49:57.979521 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:57.979453    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-66.ec2.internal.18d67b98c2c0d39a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-66.ec2.internal,UID:ip-10-0-130-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-66.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-66.ec2.internal,},FirstTimestamp:2026-09-18 17:49:57.959013274 +0000 UTC m=+2.036817164,LastTimestamp:2026-09-18 17:49:57.959013274 +0000 UTC m=+2.036817164,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-66.ec2.internal,}"
Sep 18 17:49:57.990658 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.990642    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cvsjw"
Sep 18 17:49:57.990910 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:57.990848    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-66.ec2.internal.18d67b98c2c0f90e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-66.ec2.internal,UID:ip-10-0-130-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-66.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-66.ec2.internal,},FirstTimestamp:2026-09-18 17:49:57.959022862 +0000 UTC m=+2.036826755,LastTimestamp:2026-09-18 17:49:57.959022862 +0000 UTC m=+2.036826755,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-66.ec2.internal,}"
Sep 18 17:49:57.997788 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.997775    2568 manager.go:337] "Starting Device Plugin manager"
Sep 18 17:49:58.016568 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:57.997820    2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 17:49:58.016568 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.997833    2568 server.go:87] "Starting device plugin registration server"
Sep 18 17:49:58.016568 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.998093    2568 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 17:49:58.016568 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.998116    2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 17:49:58.016568 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.998213    2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 17:49:58.016568 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.998301    2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 17:49:58.016568 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:57.998336    2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 17:49:58.016568 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:57.998959    2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 17:49:58.016568 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:57.999004    2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-66.ec2.internal\" not found"
Sep 18 17:49:58.016568 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:57.999255    2568 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-66.ec2.internal.18d67b98c2c07ac4\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-66.ec2.internal.18d67b98c2c07ac4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-66.ec2.internal,UID:ip-10-0-130-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-66.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-66.ec2.internal,},FirstTimestamp:2026-09-18 17:49:57.958990532 +0000 UTC m=+2.036794421,LastTimestamp:2026-09-18 17:49:57.960407371 +0000 UTC m=+2.038211263,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-66.ec2.internal,}"
Sep 18 17:49:58.016568 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.000380    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cvsjw"
Sep 18 17:49:58.016568 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:58.008096    2568 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-66.ec2.internal.18d67b98c2c0d39a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-66.ec2.internal.18d67b98c2c0d39a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-66.ec2.internal,UID:ip-10-0-130-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-66.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-66.ec2.internal,},FirstTimestamp:2026-09-18 17:49:57.959013274 +0000 UTC m=+2.036817164,LastTimestamp:2026-09-18 17:49:57.960424819 +0000 UTC m=+2.038228708,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-66.ec2.internal,}"
Sep 18 17:49:58.038054 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.038032    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 17:49:58.038147 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.038080    2568 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 17:49:58.038147 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.038099    2568 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 17:49:58.038147 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:58.038132    2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 17:49:58.040748 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.040732    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 17:49:58.099244 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.099214    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 17:49:58.100123 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.100105    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 17:49:58.100239 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.100136    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 17:49:58.100239 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.100148    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Sep 18 17:49:58.100239 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.100172    2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.112619 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.112598    2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.112619 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:58.112617    2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-66.ec2.internal\": node \"ip-10-0-130-66.ec2.internal\" not found"
Sep 18 17:49:58.131565 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:58.131484    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Sep 18 17:49:58.138364 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.138347    2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"]
Sep 18 17:49:58.138424 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.138405    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 17:49:58.139266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.139250    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 17:49:58.139349 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.139278    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 17:49:58.139349 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.139288    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Sep 18 17:49:58.141631 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.141620    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 17:49:58.141756 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.141741    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.141815 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.141771    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 17:49:58.142372 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.142358    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 17:49:58.142372 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.142367    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 17:49:58.142503 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.142385    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 17:49:58.142503 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.142387    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 17:49:58.142503 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.142398    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Sep 18 17:49:58.142503 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.142402    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Sep 18 17:49:58.145018 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.145001    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.145098 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.145030    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 17:49:58.145710 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.145694    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 17:49:58.145771 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.145724    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 17:49:58.145771 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.145734    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Sep 18 17:49:58.171790 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:58.171765    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-66.ec2.internal\" not found" node="ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.176081 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:58.176064    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-66.ec2.internal\" not found" node="ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.229309 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.229286    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/22239227cca568015d5adada451d696e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"22239227cca568015d5adada451d696e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.229435 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.229314    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22239227cca568015d5adada451d696e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"22239227cca568015d5adada451d696e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.229435 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.229336    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ac81c3c3c16cb65d907a4298fd5e63bf-config\") pod \"kube-apiserver-proxy-ip-10-0-130-66.ec2.internal\" (UID: \"ac81c3c3c16cb65d907a4298fd5e63bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.232286 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:58.232267    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Sep 18 17:49:58.330261 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.330230    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/22239227cca568015d5adada451d696e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"22239227cca568015d5adada451d696e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.330261 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.330265    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22239227cca568015d5adada451d696e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"22239227cca568015d5adada451d696e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.330409 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.330281    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ac81c3c3c16cb65d907a4298fd5e63bf-config\") pod \"kube-apiserver-proxy-ip-10-0-130-66.ec2.internal\" (UID: \"ac81c3c3c16cb65d907a4298fd5e63bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.330409 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.330327    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22239227cca568015d5adada451d696e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"22239227cca568015d5adada451d696e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.330409 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.330327    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/22239227cca568015d5adada451d696e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal\" (UID: \"22239227cca568015d5adada451d696e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.330409 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.330368    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ac81c3c3c16cb65d907a4298fd5e63bf-config\") pod \"kube-apiserver-proxy-ip-10-0-130-66.ec2.internal\" (UID: \"ac81c3c3c16cb65d907a4298fd5e63bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.333327 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:58.333312    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Sep 18 17:49:58.434115 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:58.434037    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Sep 18 17:49:58.475236 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.475206    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.479131 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.479110    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Sep 18 17:49:58.534523 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:58.534487    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Sep 18 17:49:58.635010 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:58.634971    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Sep 18 17:49:58.735590 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:58.735518    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Sep 18 17:49:58.755568 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.755516    2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 17:49:58.836433 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:58.836392    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Sep 18 17:49:58.881627 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.881565    2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 17:49:58.882059 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.881724    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 17:49:58.882059 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.881732    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 17:49:58.936716 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:58.936688    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Sep 18 17:49:58.987079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.987019    2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 17:49:58.997844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.997815    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 17:49:58.999855 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.999836    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 17:49:58.999934 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.999870    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 17:49:58.999934 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:58.999880    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeHasSufficientPID"
Sep 18 17:49:59.006116 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.006091    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 17:44:57 +0000 UTC" deadline="2028-04-01 19:03:37.799542718 +0000 UTC"
Sep 18 17:49:59.006116 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.006116    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13465h13m38.793429421s"
Sep 18 17:49:59.037063 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:59.037045    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Sep 18 17:49:59.137995 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:59.137826    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-66.ec2.internal\" not found"
Sep 18 17:49:59.202532 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:59.202500    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac81c3c3c16cb65d907a4298fd5e63bf.slice/crio-4de0a4c3b94c0c6258028237c62d6cc1645864ee855ec8b738926dba3483901e WatchSource:0}: Error finding container 4de0a4c3b94c0c6258028237c62d6cc1645864ee855ec8b738926dba3483901e: Status 404 returned error can't find the container with id 4de0a4c3b94c0c6258028237c62d6cc1645864ee855ec8b738926dba3483901e
Sep 18 17:49:59.202813 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:49:59.202795    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22239227cca568015d5adada451d696e.slice/crio-a2192fea6332bf134b36f6d22d55a6b1da86b4a6f292661851321431a6b4732a WatchSource:0}: Error finding container a2192fea6332bf134b36f6d22d55a6b1da86b4a6f292661851321431a6b4732a: Status 404 returned error can't find the container with id a2192fea6332bf134b36f6d22d55a6b1da86b4a6f292661851321431a6b4732a
Sep 18 17:49:59.205742 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.205718    2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 17:49:59.207570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.207556    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 17:49:59.227870 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.227839    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal"
Sep 18 17:49:59.239929 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.239873    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 17:49:59.240721 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.240708    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal"
Sep 18 17:49:59.250351 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.250330    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 17:49:59.909575 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.909531    2568 apiserver.go:52] "Watching apiserver"
Sep 18 17:49:59.915876 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.915847    2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 17:49:59.916431 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.916405    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-bc6dq","openshift-ovn-kubernetes/ovnkube-node-56mpn","kube-system/global-pull-secret-syncer-66bl2","kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh","openshift-cluster-node-tuning-operator/tuned-rqj8n","openshift-image-registry/node-ca-nhbw2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal","openshift-multus/multus-additional-cni-plugins-v8w95","openshift-multus/network-metrics-daemon-268np","kube-system/konnectivity-agent-n6fx7","openshift-dns/node-resolver-t457f","openshift-multus/multus-67z6x","openshift-network-diagnostics/network-check-target-dhmwn"]
Sep 18 17:49:59.918987 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.918967    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:49:59.919091 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:59.919068    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:49:59.921512 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.921294    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.925249 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.924416    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 17:49:59.925249 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.924645    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w6jlq\""
Sep 18 17:49:59.925249 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.924848    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 17:49:59.925249 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.925013    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 17:49:59.925249 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.925028    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 17:49:59.925593 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.925350    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:49:59.925593 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:59.925412    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:49:59.928165 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.927773    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 17:49:59.928165 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.928010    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:49:59.928165 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:49:59.928081    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:49:59.930334 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.929861    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:49:59.933055 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.932027    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-p8hm9\""
Sep 18 17:49:59.933055 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.932218    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 17:49:59.933055 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.932325    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 17:49:59.933055 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.932327    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 17:49:59.933055 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.932390    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:49:59.934532 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.934517    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 17:49:59.934815 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.934746    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 17:49:59.934815 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.934799    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p4ppk\""
Sep 18 17:49:59.935066 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.935050    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:49:59.937128 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.937110    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nhbw2"
Sep 18 17:49:59.937448 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.937429    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 17:49:59.937724 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.937513    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 17:49:59.937724 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.937432    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bjq6x\""
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.939249    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.939578    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jrzkw\""
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.939747    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.939991    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940289    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs\") pod \"network-metrics-daemon-268np\" (UID: \"71aef7ee-527e-4b8b-b17a-578530eea139\") " pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940319    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-cnibin\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940342    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-multus-socket-dir-parent\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940369    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-system-cni-dir\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940392    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-var-lib-cni-bin\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940438    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-run-netns\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940460    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-multus-conf-dir\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940492    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940518    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-sys-fs\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940547    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-etc-kubernetes\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940569    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-device-dir\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940593    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpwds\" (UniqueName: \"kubernetes.io/projected/98bc124f-b527-4aee-b192-96d7f357c5b6-kube-api-access-gpwds\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940616    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz7pv\" (UniqueName: \"kubernetes.io/projected/71aef7ee-527e-4b8b-b17a-578530eea139-kube-api-access-kz7pv\") pod \"network-metrics-daemon-268np\" (UID: \"71aef7ee-527e-4b8b-b17a-578530eea139\") " pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940641    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-run-k8s-cni-cncf-io\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.940674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940671    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-run-multus-certs\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.941795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940716    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-socket-dir\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:49:59.941795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940739    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/844bcd58-1138-4039-84dc-327f7876529d-cni-binary-copy\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.941795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940763    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-var-lib-cni-multus\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.941795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940790    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-hostroot\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.941795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940812    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2x8l\" (UniqueName: \"kubernetes.io/projected/844bcd58-1138-4039-84dc-327f7876529d-kube-api-access-q2x8l\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.941795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940835    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a5aed5d3-d115-484b-a2f9-54a887906baa-dbus\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:49:59.941795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940859    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-etc-selinux\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:49:59.941795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940879    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-multus-cni-dir\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.941795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940912    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/844bcd58-1138-4039-84dc-327f7876529d-multus-daemon-config\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.941795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940938    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a5aed5d3-d115-484b-a2f9-54a887906baa-kubelet-config\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:49:59.941795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.940978    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:49:59.941795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.941000    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-var-lib-kubelet\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:49:59.941795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.941023    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpw8p\" (UniqueName: \"kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p\") pod \"network-check-target-dhmwn\" (UID: \"e619dfc8-9394-40de-8244-b7adc5e081aa\") " pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:49:59.941795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.941048    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-registration-dir\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:49:59.941795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.941581    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 17:49:59.942676 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.941862    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:49:59.942748 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.942734    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bc6dq"
Sep 18 17:49:59.945446 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.945424    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n6fx7"
Sep 18 17:49:59.952000 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.951977    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 17:49:59.954229 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.952199    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 17:49:59.954229 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.951655    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 17:49:59.954229 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.951725    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xl4fg\""
Sep 18 17:49:59.954229 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.952476    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 17:49:59.954229 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.951770    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 17:49:59.954229 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.951899    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t457f"
Sep 18 17:49:59.954229 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.952883    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 17:49:59.954229 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.953326    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 17:49:59.954229 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.953608    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 17:49:59.954229 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.953837    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 17:49:59.954229 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.954064    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v5xp5\""
Sep 18 17:49:59.954773 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.954373    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 17:49:59.955115 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.955098    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 17:49:59.956596 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.955766    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 17:49:59.956596 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.955967    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 17:49:59.956596 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.956024    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 17:49:59.956596 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.956341    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9hbx7\""
Sep 18 17:49:59.956596 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.956350    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kn2f8\""
Sep 18 17:49:59.963324 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.963305    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-r6ksr"
Sep 18 17:49:59.970486 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:49:59.970465    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-r6ksr"
Sep 18 17:50:00.029065 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.029035    2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 17:50:00.041352 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041318    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2rf4\" (UniqueName: \"kubernetes.io/projected/968ec864-e2d4-4577-9a8e-4e63b405acfd-kube-api-access-r2rf4\") pod \"node-ca-nhbw2\" (UID: \"968ec864-e2d4-4577-9a8e-4e63b405acfd\") " pod="openshift-image-registry/node-ca-nhbw2"
Sep 18 17:50:00.041477 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041362    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bffad327-4f7a-40a1-b969-01a7799008f1-ovn-node-metrics-cert\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.041477 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041385    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-host\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.041477 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041444    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-system-cni-dir\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.041644 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041517    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-run-openvswitch\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.041644 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041546    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bbd2debc-2140-487d-ac31-a1971f9e89ee-iptables-alerter-script\") pod \"iptables-alerter-bc6dq\" (UID: \"bbd2debc-2140-487d-ac31-a1971f9e89ee\") " pod="openshift-network-operator/iptables-alerter-bc6dq"
Sep 18 17:50:00.041644 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041570    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldbsl\" (UniqueName: \"kubernetes.io/projected/885535b4-6728-4f1f-811e-8b04785fc02d-kube-api-access-ldbsl\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.041644 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041592    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-system-cni-dir\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.041644 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041599    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-multus-conf-dir\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.041845 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041646    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a2359522-605f-4dc9-9837-4b6f95ed99f6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.041845 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041646    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-multus-conf-dir\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.041845 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041675    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/968ec864-e2d4-4577-9a8e-4e63b405acfd-serviceca\") pod \"node-ca-nhbw2\" (UID: \"968ec864-e2d4-4577-9a8e-4e63b405acfd\") " pod="openshift-image-registry/node-ca-nhbw2"
Sep 18 17:50:00.041845 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041699    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-run-ovn-kubernetes\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.041845 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041721    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-sysctl-conf\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.041845 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041752    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpwds\" (UniqueName: \"kubernetes.io/projected/98bc124f-b527-4aee-b192-96d7f357c5b6-kube-api-access-gpwds\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:50:00.041845 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041778    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4ptg\" (UniqueName: \"kubernetes.io/projected/a2359522-605f-4dc9-9837-4b6f95ed99f6-kube-api-access-l4ptg\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.041845 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041816    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-cni-netd\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.041845 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041841    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb6st\" (UniqueName: \"kubernetes.io/projected/bbd2debc-2140-487d-ac31-a1971f9e89ee-kube-api-access-nb6st\") pod \"iptables-alerter-bc6dq\" (UID: \"bbd2debc-2140-487d-ac31-a1971f9e89ee\") " pod="openshift-network-operator/iptables-alerter-bc6dq"
Sep 18 17:50:00.042268 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041870    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-var-lib-kubelet\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.042268 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041900    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz7pv\" (UniqueName: \"kubernetes.io/projected/71aef7ee-527e-4b8b-b17a-578530eea139-kube-api-access-kz7pv\") pod \"network-metrics-daemon-268np\" (UID: \"71aef7ee-527e-4b8b-b17a-578530eea139\") " pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:00.042268 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041929    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-run-multus-certs\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.042268 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041955    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-socket-dir\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:50:00.042268 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.041978    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-var-lib-openvswitch\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.042268 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042001    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-var-lib-cni-multus\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.042268 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042023    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-hostroot\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.042268 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042046    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2x8l\" (UniqueName: \"kubernetes.io/projected/844bcd58-1138-4039-84dc-327f7876529d-kube-api-access-q2x8l\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.042268 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042080    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a5aed5d3-d115-484b-a2f9-54a887906baa-dbus\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:00.042268 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042250    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-hostroot\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042290    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-etc-selinux\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042319    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bd78b5f9-264b-477b-8deb-b91499c6a278-agent-certs\") pod \"konnectivity-agent-n6fx7\" (UID: \"bd78b5f9-264b-477b-8deb-b91499c6a278\") " pod="kube-system/konnectivity-agent-n6fx7"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042320    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-var-lib-cni-multus\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042343    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bffad327-4f7a-40a1-b969-01a7799008f1-env-overrides\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042370    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a5aed5d3-d115-484b-a2f9-54a887906baa-kubelet-config\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042386    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-socket-dir\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042397    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042426    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a2359522-605f-4dc9-9837-4b6f95ed99f6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042430    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-run-multus-certs\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042437    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-etc-selinux\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042474    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042493    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042505    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-var-lib-kubelet\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042534    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bd78b5f9-264b-477b-8deb-b91499c6a278-konnectivity-ca\") pod \"konnectivity-agent-n6fx7\" (UID: \"bd78b5f9-264b-477b-8deb-b91499c6a278\") " pod="kube-system/konnectivity-agent-n6fx7"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042544    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a5aed5d3-d115-484b-a2f9-54a887906baa-kubelet-config\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:00.042730 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042584    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-var-lib-kubelet\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042569    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2359522-605f-4dc9-9837-4b6f95ed99f6-system-cni-dir\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042626    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-slash\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042653    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bffad327-4f7a-40a1-b969-01a7799008f1-ovnkube-script-lib\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042665    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a5aed5d3-d115-484b-a2f9-54a887906baa-dbus\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042680    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-lib-modules\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042711    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs\") pod \"network-metrics-daemon-268np\" (UID: \"71aef7ee-527e-4b8b-b17a-578530eea139\") " pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042740    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-multus-socket-dir-parent\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042767    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-sysctl-d\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042793    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-run\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042856    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-multus-socket-dir-parent\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.042868    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042912    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-var-lib-cni-bin\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.042949    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-var-lib-cni-bin\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.042981    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs podName:71aef7ee-527e-4b8b-b17a-578530eea139 nodeName:}" failed. No retries permitted until 2026-09-18 17:50:00.542958521 +0000 UTC m=+4.620762417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs") pod "network-metrics-daemon-268np" (UID: "71aef7ee-527e-4b8b-b17a-578530eea139") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043003    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/968ec864-e2d4-4577-9a8e-4e63b405acfd-host\") pod \"node-ca-nhbw2\" (UID: \"968ec864-e2d4-4577-9a8e-4e63b405acfd\") " pod="openshift-image-registry/node-ca-nhbw2"
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043031    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bffad327-4f7a-40a1-b969-01a7799008f1-ovnkube-config\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.043401 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043053    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-sys\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043080    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-run-netns\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043103    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043126    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-sys-fs\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043153    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5654t\" (UniqueName: \"kubernetes.io/projected/85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1-kube-api-access-5654t\") pod \"node-resolver-t457f\" (UID: \"85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1\") " pod="openshift-dns/node-resolver-t457f"
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043157    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-run-netns\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043191    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-run-netns\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043217    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-cni-bin\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.043241    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043258    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-modprobe-d\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043290    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-etc-kubernetes\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.043302    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret podName:a5aed5d3-d115-484b-a2f9-54a887906baa nodeName:}" failed. No retries permitted until 2026-09-18 17:50:00.543286062 +0000 UTC m=+4.621089954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret") pod "global-pull-secret-syncer-66bl2" (UID: "a5aed5d3-d115-484b-a2f9-54a887906baa") : object "kube-system"/"original-pull-secret" not registered
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043242    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-sys-fs\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043332    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-etc-kubernetes\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043372    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-device-dir\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043400    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1-tmp-dir\") pod \"node-resolver-t457f\" (UID: \"85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1\") " pod="openshift-dns/node-resolver-t457f"
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043436    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-run-k8s-cni-cncf-io\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.044079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043457    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-device-dir\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043461    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-kubelet\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043484    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/885535b4-6728-4f1f-811e-8b04785fc02d-tmp\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043504    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/844bcd58-1138-4039-84dc-327f7876529d-cni-binary-copy\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043530    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2359522-605f-4dc9-9837-4b6f95ed99f6-cnibin\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043500    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-host-run-k8s-cni-cncf-io\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043558    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2359522-605f-4dc9-9837-4b6f95ed99f6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043585    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-etc-openvswitch\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043609    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khr6p\" (UniqueName: \"kubernetes.io/projected/bffad327-4f7a-40a1-b969-01a7799008f1-kube-api-access-khr6p\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043632    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bbd2debc-2140-487d-ac31-a1971f9e89ee-host-slash\") pod \"iptables-alerter-bc6dq\" (UID: \"bbd2debc-2140-487d-ac31-a1971f9e89ee\") " pod="openshift-network-operator/iptables-alerter-bc6dq"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043672    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-multus-cni-dir\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043697    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/844bcd58-1138-4039-84dc-327f7876529d-multus-daemon-config\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043722    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1-hosts-file\") pod \"node-resolver-t457f\" (UID: \"85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1\") " pod="openshift-dns/node-resolver-t457f"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043746    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-systemd-units\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043769    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-run-ovn\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043791    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-node-log\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043816    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-log-socket\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.044802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043861    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-kubernetes\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.045570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043888    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpw8p\" (UniqueName: \"kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p\") pod \"network-check-target-dhmwn\" (UID: \"e619dfc8-9394-40de-8244-b7adc5e081aa\") " pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:00.045570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043914    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-registration-dir\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:50:00.045570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043930    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2359522-605f-4dc9-9837-4b6f95ed99f6-cni-binary-copy\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.045570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043951    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-run-systemd\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.045570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043974    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-sysconfig\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.045570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.043997    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-systemd\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.045570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.044019    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/885535b4-6728-4f1f-811e-8b04785fc02d-etc-tuned\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.045570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.044019    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal" event={"ID":"ac81c3c3c16cb65d907a4298fd5e63bf","Type":"ContainerStarted","Data":"4de0a4c3b94c0c6258028237c62d6cc1645864ee855ec8b738926dba3483901e"}
Sep 18 17:50:00.045570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.044060    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-cnibin\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.045570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.044096    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/844bcd58-1138-4039-84dc-327f7876529d-cni-binary-copy\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.045570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.044150    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/844bcd58-1138-4039-84dc-327f7876529d-multus-daemon-config\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.045570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.044159    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-cnibin\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.045570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.044162    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/844bcd58-1138-4039-84dc-327f7876529d-multus-cni-dir\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.045570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.044263    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/98bc124f-b527-4aee-b192-96d7f357c5b6-registration-dir\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:50:00.046391 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.046366    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal" event={"ID":"22239227cca568015d5adada451d696e","Type":"ContainerStarted","Data":"a2192fea6332bf134b36f6d22d55a6b1da86b4a6f292661851321431a6b4732a"}
Sep 18 17:50:00.054029 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.053998    2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 17:50:00.054601 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.054580    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 17:50:00.054691 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.054615    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 17:50:00.054691 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.054628    2568 projected.go:196] Error preparing data for projected volume kube-api-access-lpw8p for pod openshift-network-diagnostics/network-check-target-dhmwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 17:50:00.054691 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.054685    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p podName:e619dfc8-9394-40de-8244-b7adc5e081aa nodeName:}" failed. No retries permitted until 2026-09-18 17:50:00.554668327 +0000 UTC m=+4.632472225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lpw8p" (UniqueName: "kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p") pod "network-check-target-dhmwn" (UID: "e619dfc8-9394-40de-8244-b7adc5e081aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 17:50:00.060141 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.059860    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpwds\" (UniqueName: \"kubernetes.io/projected/98bc124f-b527-4aee-b192-96d7f357c5b6-kube-api-access-gpwds\") pod \"aws-ebs-csi-driver-node-wqhmh\" (UID: \"98bc124f-b527-4aee-b192-96d7f357c5b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:50:00.061640 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.061617    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2x8l\" (UniqueName: \"kubernetes.io/projected/844bcd58-1138-4039-84dc-327f7876529d-kube-api-access-q2x8l\") pod \"multus-67z6x\" (UID: \"844bcd58-1138-4039-84dc-327f7876529d\") " pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.066439 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.066416    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz7pv\" (UniqueName: \"kubernetes.io/projected/71aef7ee-527e-4b8b-b17a-578530eea139-kube-api-access-kz7pv\") pod \"network-metrics-daemon-268np\" (UID: \"71aef7ee-527e-4b8b-b17a-578530eea139\") " pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:00.145167 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145126    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5654t\" (UniqueName: \"kubernetes.io/projected/85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1-kube-api-access-5654t\") pod \"node-resolver-t457f\" (UID: \"85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1\") " pod="openshift-dns/node-resolver-t457f"
Sep 18 17:50:00.145362 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145171    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-run-netns\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.145362 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145214    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-cni-bin\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.145362 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145240    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-modprobe-d\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.145362 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145251    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-run-netns\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.145362 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145268    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1-tmp-dir\") pod \"node-resolver-t457f\" (UID: \"85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1\") " pod="openshift-dns/node-resolver-t457f"
Sep 18 17:50:00.145362 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145292    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-kubelet\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.145362 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145312    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/885535b4-6728-4f1f-811e-8b04785fc02d-tmp\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.145362 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145338    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2359522-605f-4dc9-9837-4b6f95ed99f6-cnibin\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.145362 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145362    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2359522-605f-4dc9-9837-4b6f95ed99f6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145388    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-etc-openvswitch\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145395    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-modprobe-d\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145413    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khr6p\" (UniqueName: \"kubernetes.io/projected/bffad327-4f7a-40a1-b969-01a7799008f1-kube-api-access-khr6p\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145437    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bbd2debc-2140-487d-ac31-a1971f9e89ee-host-slash\") pod \"iptables-alerter-bc6dq\" (UID: \"bbd2debc-2140-487d-ac31-a1971f9e89ee\") " pod="openshift-network-operator/iptables-alerter-bc6dq"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145444    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-cni-bin\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145466    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1-hosts-file\") pod \"node-resolver-t457f\" (UID: \"85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1\") " pod="openshift-dns/node-resolver-t457f"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145506    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-systemd-units\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145530    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-run-ovn\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145552    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-node-log\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145574    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-log-socket\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145596    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-kubernetes\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145634    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2359522-605f-4dc9-9837-4b6f95ed99f6-cni-binary-copy\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145661    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-run-systemd\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145684    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-sysconfig\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145706    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-systemd\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145728    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/885535b4-6728-4f1f-811e-8b04785fc02d-etc-tuned\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145750    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1-tmp-dir\") pod \"node-resolver-t457f\" (UID: \"85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1\") " pod="openshift-dns/node-resolver-t457f"
Sep 18 17:50:00.145844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145751    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2rf4\" (UniqueName: \"kubernetes.io/projected/968ec864-e2d4-4577-9a8e-4e63b405acfd-kube-api-access-r2rf4\") pod \"node-ca-nhbw2\" (UID: \"968ec864-e2d4-4577-9a8e-4e63b405acfd\") " pod="openshift-image-registry/node-ca-nhbw2"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145839    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bffad327-4f7a-40a1-b969-01a7799008f1-ovn-node-metrics-cert\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145857    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-kubelet\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145866    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-host\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145900    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-run-openvswitch\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145914    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-host\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145929    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bbd2debc-2140-487d-ac31-a1971f9e89ee-iptables-alerter-script\") pod \"iptables-alerter-bc6dq\" (UID: \"bbd2debc-2140-487d-ac31-a1971f9e89ee\") " pod="openshift-network-operator/iptables-alerter-bc6dq"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145954    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldbsl\" (UniqueName: \"kubernetes.io/projected/885535b4-6728-4f1f-811e-8b04785fc02d-kube-api-access-ldbsl\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145995    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2359522-605f-4dc9-9837-4b6f95ed99f6-cnibin\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.146035    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-run-openvswitch\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.146523    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-node-log\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.146580    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bbd2debc-2140-487d-ac31-a1971f9e89ee-iptables-alerter-script\") pod \"iptables-alerter-bc6dq\" (UID: \"bbd2debc-2140-487d-ac31-a1971f9e89ee\") " pod="openshift-network-operator/iptables-alerter-bc6dq"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.146580    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a2359522-605f-4dc9-9837-4b6f95ed99f6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.146630    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-log-socket\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.146661    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-sysconfig\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.146675    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2359522-605f-4dc9-9837-4b6f95ed99f6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.146690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.146682    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-systemd\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.146722    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-etc-openvswitch\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.146725    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-kubernetes\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.146805    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-run-systemd\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.145999    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a2359522-605f-4dc9-9837-4b6f95ed99f6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.146973    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/968ec864-e2d4-4577-9a8e-4e63b405acfd-serviceca\") pod \"node-ca-nhbw2\" (UID: \"968ec864-e2d4-4577-9a8e-4e63b405acfd\") " pod="openshift-image-registry/node-ca-nhbw2"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147000    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-run-ovn-kubernetes\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147041    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-sysctl-conf\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147069    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4ptg\" (UniqueName: \"kubernetes.io/projected/a2359522-605f-4dc9-9837-4b6f95ed99f6-kube-api-access-l4ptg\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147110    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-cni-netd\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147135    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb6st\" (UniqueName: \"kubernetes.io/projected/bbd2debc-2140-487d-ac31-a1971f9e89ee-kube-api-access-nb6st\") pod \"iptables-alerter-bc6dq\" (UID: \"bbd2debc-2140-487d-ac31-a1971f9e89ee\") " pod="openshift-network-operator/iptables-alerter-bc6dq"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147158    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-var-lib-kubelet\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147204    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-var-lib-openvswitch\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147236    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bd78b5f9-264b-477b-8deb-b91499c6a278-agent-certs\") pod \"konnectivity-agent-n6fx7\" (UID: \"bd78b5f9-264b-477b-8deb-b91499c6a278\") " pod="kube-system/konnectivity-agent-n6fx7"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147259    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bffad327-4f7a-40a1-b969-01a7799008f1-env-overrides\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147284    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a2359522-605f-4dc9-9837-4b6f95ed99f6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147311    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.147437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147337    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bd78b5f9-264b-477b-8deb-b91499c6a278-konnectivity-ca\") pod \"konnectivity-agent-n6fx7\" (UID: \"bd78b5f9-264b-477b-8deb-b91499c6a278\") " pod="kube-system/konnectivity-agent-n6fx7"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147362    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2359522-605f-4dc9-9837-4b6f95ed99f6-system-cni-dir\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147388    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-slash\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147403    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-var-lib-kubelet\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147204    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2359522-605f-4dc9-9837-4b6f95ed99f6-cni-binary-copy\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147436    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bbd2debc-2140-487d-ac31-a1971f9e89ee-host-slash\") pod \"iptables-alerter-bc6dq\" (UID: \"bbd2debc-2140-487d-ac31-a1971f9e89ee\") " pod="openshift-network-operator/iptables-alerter-bc6dq"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147465    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-run-ovn-kubernetes\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147475    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bffad327-4f7a-40a1-b969-01a7799008f1-ovnkube-script-lib\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147527    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-lib-modules\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147565    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-sysctl-d\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147593    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-run\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147618    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/968ec864-e2d4-4577-9a8e-4e63b405acfd-host\") pod \"node-ca-nhbw2\" (UID: \"968ec864-e2d4-4577-9a8e-4e63b405acfd\") " pod="openshift-image-registry/node-ca-nhbw2"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147641    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bffad327-4f7a-40a1-b969-01a7799008f1-ovnkube-config\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147666    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-sys\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147752    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-sys\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147867    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-sysctl-conf\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.147906    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/968ec864-e2d4-4577-9a8e-4e63b405acfd-serviceca\") pod \"node-ca-nhbw2\" (UID: \"968ec864-e2d4-4577-9a8e-4e63b405acfd\") " pod="openshift-image-registry/node-ca-nhbw2"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.148023    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bffad327-4f7a-40a1-b969-01a7799008f1-ovnkube-script-lib\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.148266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.148091    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1-hosts-file\") pod \"node-resolver-t457f\" (UID: \"85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1\") " pod="openshift-dns/node-resolver-t457f"
Sep 18 17:50:00.149106 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.148101    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-cni-netd\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.149454 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.148133    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-systemd-units\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.149725 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.149705    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-run-ovn\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.151731 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.151706    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bffad327-4f7a-40a1-b969-01a7799008f1-ovn-node-metrics-cert\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.151956 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.151936    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/885535b4-6728-4f1f-811e-8b04785fc02d-etc-tuned\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.152384 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.152363    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bffad327-4f7a-40a1-b969-01a7799008f1-env-overrides\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.152864 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.152842    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a2359522-605f-4dc9-9837-4b6f95ed99f6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.152945 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.152928    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.153486 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.153464    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bd78b5f9-264b-477b-8deb-b91499c6a278-konnectivity-ca\") pod \"konnectivity-agent-n6fx7\" (UID: \"bd78b5f9-264b-477b-8deb-b91499c6a278\") " pod="kube-system/konnectivity-agent-n6fx7"
Sep 18 17:50:00.153561 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.153544    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2359522-605f-4dc9-9837-4b6f95ed99f6-system-cni-dir\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.153632 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.153594    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-var-lib-openvswitch\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.154170 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.154149    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-lib-modules\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.154334 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.154314    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-etc-sysctl-d\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.154398 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.154383    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/885535b4-6728-4f1f-811e-8b04785fc02d-run\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.154449 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.154439    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/968ec864-e2d4-4577-9a8e-4e63b405acfd-host\") pod \"node-ca-nhbw2\" (UID: \"968ec864-e2d4-4577-9a8e-4e63b405acfd\") " pod="openshift-image-registry/node-ca-nhbw2"
Sep 18 17:50:00.154806 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.154786    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bffad327-4f7a-40a1-b969-01a7799008f1-host-slash\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.154942 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.154920    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bffad327-4f7a-40a1-b969-01a7799008f1-ovnkube-config\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.154994 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.154979    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/885535b4-6728-4f1f-811e-8b04785fc02d-tmp\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.156571 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.156548    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bd78b5f9-264b-477b-8deb-b91499c6a278-agent-certs\") pod \"konnectivity-agent-n6fx7\" (UID: \"bd78b5f9-264b-477b-8deb-b91499c6a278\") " pod="kube-system/konnectivity-agent-n6fx7"
Sep 18 17:50:00.157707 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.157688    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5654t\" (UniqueName: \"kubernetes.io/projected/85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1-kube-api-access-5654t\") pod \"node-resolver-t457f\" (UID: \"85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1\") " pod="openshift-dns/node-resolver-t457f"
Sep 18 17:50:00.192324 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.192295    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb6st\" (UniqueName: \"kubernetes.io/projected/bbd2debc-2140-487d-ac31-a1971f9e89ee-kube-api-access-nb6st\") pod \"iptables-alerter-bc6dq\" (UID: \"bbd2debc-2140-487d-ac31-a1971f9e89ee\") " pod="openshift-network-operator/iptables-alerter-bc6dq"
Sep 18 17:50:00.192722 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.192700    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4ptg\" (UniqueName: \"kubernetes.io/projected/a2359522-605f-4dc9-9837-4b6f95ed99f6-kube-api-access-l4ptg\") pod \"multus-additional-cni-plugins-v8w95\" (UID: \"a2359522-605f-4dc9-9837-4b6f95ed99f6\") " pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.192808 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.192762    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldbsl\" (UniqueName: \"kubernetes.io/projected/885535b4-6728-4f1f-811e-8b04785fc02d-kube-api-access-ldbsl\") pod \"tuned-rqj8n\" (UID: \"885535b4-6728-4f1f-811e-8b04785fc02d\") " pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.193242 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.193223    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2rf4\" (UniqueName: \"kubernetes.io/projected/968ec864-e2d4-4577-9a8e-4e63b405acfd-kube-api-access-r2rf4\") pod \"node-ca-nhbw2\" (UID: \"968ec864-e2d4-4577-9a8e-4e63b405acfd\") " pod="openshift-image-registry/node-ca-nhbw2"
Sep 18 17:50:00.193637 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.193613    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khr6p\" (UniqueName: \"kubernetes.io/projected/bffad327-4f7a-40a1-b969-01a7799008f1-kube-api-access-khr6p\") pod \"ovnkube-node-56mpn\" (UID: \"bffad327-4f7a-40a1-b969-01a7799008f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.233704 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.233673    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-67z6x"
Sep 18 17:50:00.253270 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.253235    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh"
Sep 18 17:50:00.265278 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.265255    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nhbw2"
Sep 18 17:50:00.274031 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.274006    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rqj8n"
Sep 18 17:50:00.281847 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.281828    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v8w95"
Sep 18 17:50:00.290575 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.290551    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:00.306145 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.306117    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bc6dq"
Sep 18 17:50:00.312899 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.312874    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n6fx7"
Sep 18 17:50:00.319548 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.319528    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t457f"
Sep 18 17:50:00.550732 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.550646    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs\") pod \"network-metrics-daemon-268np\" (UID: \"71aef7ee-527e-4b8b-b17a-578530eea139\") " pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:00.550732 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.550684    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:00.550962 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.550806    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 17:50:00.550962 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.550868    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs podName:71aef7ee-527e-4b8b-b17a-578530eea139 nodeName:}" failed. No retries permitted until 2026-09-18 17:50:01.55085296 +0000 UTC m=+5.628656836 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs") pod "network-metrics-daemon-268np" (UID: "71aef7ee-527e-4b8b-b17a-578530eea139") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 17:50:00.550962 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.550806    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 17:50:00.551119 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.550988    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret podName:a5aed5d3-d115-484b-a2f9-54a887906baa nodeName:}" failed. No retries permitted until 2026-09-18 17:50:01.55096736 +0000 UTC m=+5.628771256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret") pod "global-pull-secret-syncer-66bl2" (UID: "a5aed5d3-d115-484b-a2f9-54a887906baa") : object "kube-system"/"original-pull-secret" not registered
Sep 18 17:50:00.651675 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.651638    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpw8p\" (UniqueName: \"kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p\") pod \"network-check-target-dhmwn\" (UID: \"e619dfc8-9394-40de-8244-b7adc5e081aa\") " pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:00.651865 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.651798    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 17:50:00.651865 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.651819    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 17:50:00.651865 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.651829    2568 projected.go:196] Error preparing data for projected volume kube-api-access-lpw8p for pod openshift-network-diagnostics/network-check-target-dhmwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 17:50:00.652021 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:00.651891    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p podName:e619dfc8-9394-40de-8244-b7adc5e081aa nodeName:}" failed. No retries permitted until 2026-09-18 17:50:01.651877321 +0000 UTC m=+5.729681198 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpw8p" (UniqueName: "kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p") pod "network-check-target-dhmwn" (UID: "e619dfc8-9394-40de-8244-b7adc5e081aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 17:50:00.974313 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.974228    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 17:44:59 +0000 UTC" deadline="2028-02-25 01:14:34.239670686 +0000 UTC"
Sep 18 17:50:00.974313 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:00.974258    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12583h24m33.265417218s"
Sep 18 17:50:01.031529 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:01.031496    2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 17:50:01.038682 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:01.038654    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:01.038812 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:01.038708    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:01.038812 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:01.038775    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:01.038959 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:01.038934    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:01.167495 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:50:01.167456    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ae4e18_9ca4_44dc_9009_4c8d9e4c82e1.slice/crio-21fbcf5022b4f7ded310547790531333970d2c03e7aff53c07d11ea4d9624f31 WatchSource:0}: Error finding container 21fbcf5022b4f7ded310547790531333970d2c03e7aff53c07d11ea4d9624f31: Status 404 returned error can't find the container with id 21fbcf5022b4f7ded310547790531333970d2c03e7aff53c07d11ea4d9624f31
Sep 18 17:50:01.168792 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:50:01.168755    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbffad327_4f7a_40a1_b969_01a7799008f1.slice/crio-1968ecef5cf912eef8658dc603ba69334a0625271cf9186e7409758da56227a5 WatchSource:0}: Error finding container 1968ecef5cf912eef8658dc603ba69334a0625271cf9186e7409758da56227a5: Status 404 returned error can't find the container with id 1968ecef5cf912eef8658dc603ba69334a0625271cf9186e7409758da56227a5
Sep 18 17:50:01.169923 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:50:01.169882    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2359522_605f_4dc9_9837_4b6f95ed99f6.slice/crio-24989129087d08c5e54e7e17dbc52f2ff1d267b3832d1517cb324525282e3447 WatchSource:0}: Error finding container 24989129087d08c5e54e7e17dbc52f2ff1d267b3832d1517cb324525282e3447: Status 404 returned error can't find the container with id 24989129087d08c5e54e7e17dbc52f2ff1d267b3832d1517cb324525282e3447
Sep 18 17:50:01.170727 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:50:01.170703    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod844bcd58_1138_4039_84dc_327f7876529d.slice/crio-7a86ceb4da644dcc1271fa1d3b364d26d3d90cc85b9691ea75fcbd29a19b1e7b WatchSource:0}: Error finding container 7a86ceb4da644dcc1271fa1d3b364d26d3d90cc85b9691ea75fcbd29a19b1e7b: Status 404 returned error can't find the container with id 7a86ceb4da644dcc1271fa1d3b364d26d3d90cc85b9691ea75fcbd29a19b1e7b
Sep 18 17:50:01.173608 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:50:01.172557    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd78b5f9_264b_477b_8deb_b91499c6a278.slice/crio-524d14881e912c1ef302bedd139b2266b1ea7c5cec8071fde8f175117b6b7540 WatchSource:0}: Error finding container 524d14881e912c1ef302bedd139b2266b1ea7c5cec8071fde8f175117b6b7540: Status 404 returned error can't find the container with id 524d14881e912c1ef302bedd139b2266b1ea7c5cec8071fde8f175117b6b7540
Sep 18 17:50:01.173608 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:50:01.173518    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98bc124f_b527_4aee_b192_96d7f357c5b6.slice/crio-606e96df7da114b2b25662fb945d510f23f697087d26e17d711a7f385b618df6 WatchSource:0}: Error finding container 606e96df7da114b2b25662fb945d510f23f697087d26e17d711a7f385b618df6: Status 404 returned error can't find the container with id 606e96df7da114b2b25662fb945d510f23f697087d26e17d711a7f385b618df6
Sep 18 17:50:01.201412 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:50:01.201381    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod968ec864_e2d4_4577_9a8e_4e63b405acfd.slice/crio-0f871a8db9524a3ad623b48a30882077d88a6190e9c092252dadbc49fc73029b WatchSource:0}: Error finding container 0f871a8db9524a3ad623b48a30882077d88a6190e9c092252dadbc49fc73029b: Status 404 returned error can't find the container with id 0f871a8db9524a3ad623b48a30882077d88a6190e9c092252dadbc49fc73029b
Sep 18 17:50:01.202069 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:50:01.202044    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbd2debc_2140_487d_ac31_a1971f9e89ee.slice/crio-e173e07b053adfad2285af1d5be95a5d7be871939e0970949bb892cdaa3b221d WatchSource:0}: Error finding container e173e07b053adfad2285af1d5be95a5d7be871939e0970949bb892cdaa3b221d: Status 404 returned error can't find the container with id e173e07b053adfad2285af1d5be95a5d7be871939e0970949bb892cdaa3b221d
Sep 18 17:50:01.238792 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:50:01.238711    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod885535b4_6728_4f1f_811e_8b04785fc02d.slice/crio-bb9717f4c9af36750a8278a792b227cca95e0f87078e54a581b0bde3309f9f10 WatchSource:0}: Error finding container bb9717f4c9af36750a8278a792b227cca95e0f87078e54a581b0bde3309f9f10: Status 404 returned error can't find the container with id bb9717f4c9af36750a8278a792b227cca95e0f87078e54a581b0bde3309f9f10
Sep 18 17:50:01.558164 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:01.558058    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs\") pod \"network-metrics-daemon-268np\" (UID: \"71aef7ee-527e-4b8b-b17a-578530eea139\") " pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:01.558164 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:01.558101    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:01.558374 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:01.558214    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 17:50:01.558374 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:01.558225    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 17:50:01.558374 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:01.558267    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret podName:a5aed5d3-d115-484b-a2f9-54a887906baa nodeName:}" failed. No retries permitted until 2026-09-18 17:50:03.558254498 +0000 UTC m=+7.636058374 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret") pod "global-pull-secret-syncer-66bl2" (UID: "a5aed5d3-d115-484b-a2f9-54a887906baa") : object "kube-system"/"original-pull-secret" not registered
Sep 18 17:50:01.558374 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:01.558281    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs podName:71aef7ee-527e-4b8b-b17a-578530eea139 nodeName:}" failed. No retries permitted until 2026-09-18 17:50:03.558274992 +0000 UTC m=+7.636078869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs") pod "network-metrics-daemon-268np" (UID: "71aef7ee-527e-4b8b-b17a-578530eea139") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 17:50:01.658673 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:01.658629    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpw8p\" (UniqueName: \"kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p\") pod \"network-check-target-dhmwn\" (UID: \"e619dfc8-9394-40de-8244-b7adc5e081aa\") " pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:01.658858 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:01.658765    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 17:50:01.658858 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:01.658785    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 17:50:01.658858 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:01.658796    2568 projected.go:196] Error preparing data for projected volume kube-api-access-lpw8p for pod openshift-network-diagnostics/network-check-target-dhmwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 17:50:01.658858 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:01.658849    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p podName:e619dfc8-9394-40de-8244-b7adc5e081aa nodeName:}" failed. No retries permitted until 2026-09-18 17:50:03.658833807 +0000 UTC m=+7.736637685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpw8p" (UniqueName: "kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p") pod "network-check-target-dhmwn" (UID: "e619dfc8-9394-40de-8244-b7adc5e081aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 17:50:01.974786 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:01.974679    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 17:44:59 +0000 UTC" deadline="2028-03-31 17:20:15.068981314 +0000 UTC"
Sep 18 17:50:01.974786 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:01.974728    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13439h30m13.094260634s"
Sep 18 17:50:02.040572 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:02.040392    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:02.040572 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:02.040536    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:02.082865 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:02.082394    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal" event={"ID":"ac81c3c3c16cb65d907a4298fd5e63bf","Type":"ContainerStarted","Data":"30c95e8efa1cccf4eb66ce2228c778494c702ac1ff6bcdcb0f3f139f5bada391"}
Sep 18 17:50:02.095526 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:02.094805    2568 generic.go:356] "Generic (PLEG): container finished" podID="22239227cca568015d5adada451d696e" containerID="c6551b0fc2895f2779cfd3c7f77b4819a2715d1596930fc0d359fb7291c9617a" exitCode=0
Sep 18 17:50:02.095526 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:02.095452    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal" event={"ID":"22239227cca568015d5adada451d696e","Type":"ContainerDied","Data":"c6551b0fc2895f2779cfd3c7f77b4819a2715d1596930fc0d359fb7291c9617a"}
Sep 18 17:50:02.100500 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:02.099704    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-66.ec2.internal" podStartSLOduration=3.099685809 podStartE2EDuration="3.099685809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 17:49:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:50:02.099095733 +0000 UTC m=+6.176899635" watchObservedRunningTime="2026-09-18 17:50:02.099685809 +0000 UTC m=+6.177489710"
Sep 18 17:50:02.100500 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:02.100083    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bc6dq" event={"ID":"bbd2debc-2140-487d-ac31-a1971f9e89ee","Type":"ContainerStarted","Data":"e173e07b053adfad2285af1d5be95a5d7be871939e0970949bb892cdaa3b221d"}
Sep 18 17:50:02.102321 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:02.102097    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh" event={"ID":"98bc124f-b527-4aee-b192-96d7f357c5b6","Type":"ContainerStarted","Data":"606e96df7da114b2b25662fb945d510f23f697087d26e17d711a7f385b618df6"}
Sep 18 17:50:02.107474 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:02.107387    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn" event={"ID":"bffad327-4f7a-40a1-b969-01a7799008f1","Type":"ContainerStarted","Data":"1968ecef5cf912eef8658dc603ba69334a0625271cf9186e7409758da56227a5"}
Sep 18 17:50:02.132940 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:02.132592    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t457f" event={"ID":"85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1","Type":"ContainerStarted","Data":"21fbcf5022b4f7ded310547790531333970d2c03e7aff53c07d11ea4d9624f31"}
Sep 18 17:50:02.139927 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:02.139888    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rqj8n" event={"ID":"885535b4-6728-4f1f-811e-8b04785fc02d","Type":"ContainerStarted","Data":"bb9717f4c9af36750a8278a792b227cca95e0f87078e54a581b0bde3309f9f10"}
Sep 18 17:50:02.159594 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:02.159556    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nhbw2" event={"ID":"968ec864-e2d4-4577-9a8e-4e63b405acfd","Type":"ContainerStarted","Data":"0f871a8db9524a3ad623b48a30882077d88a6190e9c092252dadbc49fc73029b"}
Sep 18 17:50:02.173116 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:02.173076    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8w95" event={"ID":"a2359522-605f-4dc9-9837-4b6f95ed99f6","Type":"ContainerStarted","Data":"24989129087d08c5e54e7e17dbc52f2ff1d267b3832d1517cb324525282e3447"}
Sep 18 17:50:02.194714 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:02.194624    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n6fx7" event={"ID":"bd78b5f9-264b-477b-8deb-b91499c6a278","Type":"ContainerStarted","Data":"524d14881e912c1ef302bedd139b2266b1ea7c5cec8071fde8f175117b6b7540"}
Sep 18 17:50:02.205078 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:02.205035    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-67z6x" event={"ID":"844bcd58-1138-4039-84dc-327f7876529d","Type":"ContainerStarted","Data":"7a86ceb4da644dcc1271fa1d3b364d26d3d90cc85b9691ea75fcbd29a19b1e7b"}
Sep 18 17:50:03.041413 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:03.041377    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:03.041864 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:03.041521    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:03.041992 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:03.041969    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:03.042113 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:03.042090    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:03.221023 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:03.220949    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal" event={"ID":"22239227cca568015d5adada451d696e","Type":"ContainerStarted","Data":"9c1fa38be685f4aebf0ae3aa7bbd4e06868a9d002615133dc378496ee2e08747"}
Sep 18 17:50:03.576498 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:03.576457    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs\") pod \"network-metrics-daemon-268np\" (UID: \"71aef7ee-527e-4b8b-b17a-578530eea139\") " pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:03.576668 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:03.576514    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:03.576668 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:03.576647    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 17:50:03.576798 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:03.576708    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret podName:a5aed5d3-d115-484b-a2f9-54a887906baa nodeName:}" failed. No retries permitted until 2026-09-18 17:50:07.576690748 +0000 UTC m=+11.654494630 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret") pod "global-pull-secret-syncer-66bl2" (UID: "a5aed5d3-d115-484b-a2f9-54a887906baa") : object "kube-system"/"original-pull-secret" not registered
Sep 18 17:50:03.578500 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:03.577119    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 17:50:03.578500 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:03.577170    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs podName:71aef7ee-527e-4b8b-b17a-578530eea139 nodeName:}" failed. No retries permitted until 2026-09-18 17:50:07.577155371 +0000 UTC m=+11.654959256 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs") pod "network-metrics-daemon-268np" (UID: "71aef7ee-527e-4b8b-b17a-578530eea139") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 17:50:03.677071 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:03.677028    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpw8p\" (UniqueName: \"kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p\") pod \"network-check-target-dhmwn\" (UID: \"e619dfc8-9394-40de-8244-b7adc5e081aa\") " pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:03.677271 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:03.677220    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 17:50:03.677271 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:03.677245    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 17:50:03.677271 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:03.677257    2568 projected.go:196] Error preparing data for projected volume kube-api-access-lpw8p for pod openshift-network-diagnostics/network-check-target-dhmwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 17:50:03.677433 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:03.677326    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p podName:e619dfc8-9394-40de-8244-b7adc5e081aa nodeName:}" failed. No retries permitted until 2026-09-18 17:50:07.677306275 +0000 UTC m=+11.755110152 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpw8p" (UniqueName: "kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p") pod "network-check-target-dhmwn" (UID: "e619dfc8-9394-40de-8244-b7adc5e081aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 17:50:04.041539 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:04.041464    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:04.041961 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:04.041610    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:05.039198 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:05.038623    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:05.039198 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:05.038623    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:05.039198 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:05.038777    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:05.039198 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:05.038878    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:06.038825 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:06.038671    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:06.038825 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:06.038820    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:07.039354 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:07.039308    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:07.039895 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:07.039463    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:07.039895 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:07.039495    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:07.039895 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:07.039665    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:07.609564 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:07.608752    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs\") pod \"network-metrics-daemon-268np\" (UID: \"71aef7ee-527e-4b8b-b17a-578530eea139\") " pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:07.609564 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:07.608813    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:07.609564 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:07.608949    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 17:50:07.609564 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:07.609015    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret podName:a5aed5d3-d115-484b-a2f9-54a887906baa nodeName:}" failed. No retries permitted until 2026-09-18 17:50:15.608995654 +0000 UTC m=+19.686799554 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret") pod "global-pull-secret-syncer-66bl2" (UID: "a5aed5d3-d115-484b-a2f9-54a887906baa") : object "kube-system"/"original-pull-secret" not registered
Sep 18 17:50:07.609564 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:07.609456    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 17:50:07.609564 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:07.609510    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs podName:71aef7ee-527e-4b8b-b17a-578530eea139 nodeName:}" failed. No retries permitted until 2026-09-18 17:50:15.609493391 +0000 UTC m=+19.687297275 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs") pod "network-metrics-daemon-268np" (UID: "71aef7ee-527e-4b8b-b17a-578530eea139") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 17:50:07.709727 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:07.709685    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpw8p\" (UniqueName: \"kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p\") pod \"network-check-target-dhmwn\" (UID: \"e619dfc8-9394-40de-8244-b7adc5e081aa\") " pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:07.709942 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:07.709917    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 17:50:07.710022 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:07.709945    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 17:50:07.710022 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:07.709958    2568 projected.go:196] Error preparing data for projected volume kube-api-access-lpw8p for pod openshift-network-diagnostics/network-check-target-dhmwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 17:50:07.710125 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:07.710022    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p podName:e619dfc8-9394-40de-8244-b7adc5e081aa nodeName:}" failed. No retries permitted until 2026-09-18 17:50:15.710003106 +0000 UTC m=+19.787807006 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpw8p" (UniqueName: "kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p") pod "network-check-target-dhmwn" (UID: "e619dfc8-9394-40de-8244-b7adc5e081aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 17:50:08.041938 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:08.041448    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:08.041938 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:08.041556    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:09.039610 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:09.038937    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:09.039610 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:09.039076    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:09.039610 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:09.039471    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:09.039610 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:09.039575    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:10.039537 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:10.039419    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:10.040028 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:10.039568    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:11.038498 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:11.038460    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:11.038675 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:11.038507    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:11.038675 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:11.038601    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:11.038808 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:11.038733    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:12.039356 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:12.039312    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:12.039899 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:12.039467    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:13.039219 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:13.039167    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:13.039399 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:13.039204    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:13.039399 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:13.039324    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:13.039399 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:13.039346    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:14.038847 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:14.038819    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:14.039025 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:14.038942    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:15.039051 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:15.038959    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:15.039051 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:15.038996    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:15.039587 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:15.039108    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:15.039587 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:15.039233    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:15.667108 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:15.667072    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs\") pod \"network-metrics-daemon-268np\" (UID: \"71aef7ee-527e-4b8b-b17a-578530eea139\") " pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:15.667108 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:15.667110    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:15.667338 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:15.667225    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 17:50:15.667338 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:15.667237    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 17:50:15.667338 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:15.667286    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret podName:a5aed5d3-d115-484b-a2f9-54a887906baa nodeName:}" failed. No retries permitted until 2026-09-18 17:50:31.667270175 +0000 UTC m=+35.745074055 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret") pod "global-pull-secret-syncer-66bl2" (UID: "a5aed5d3-d115-484b-a2f9-54a887906baa") : object "kube-system"/"original-pull-secret" not registered
Sep 18 17:50:15.667338 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:15.667299    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs podName:71aef7ee-527e-4b8b-b17a-578530eea139 nodeName:}" failed. No retries permitted until 2026-09-18 17:50:31.667293348 +0000 UTC m=+35.745097224 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs") pod "network-metrics-daemon-268np" (UID: "71aef7ee-527e-4b8b-b17a-578530eea139") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 17:50:15.767986 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:15.767954    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpw8p\" (UniqueName: \"kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p\") pod \"network-check-target-dhmwn\" (UID: \"e619dfc8-9394-40de-8244-b7adc5e081aa\") " pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:15.768163 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:15.768134    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 17:50:15.768163 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:15.768154    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 17:50:15.768287 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:15.768167    2568 projected.go:196] Error preparing data for projected volume kube-api-access-lpw8p for pod openshift-network-diagnostics/network-check-target-dhmwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 17:50:15.768287 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:15.768234    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p podName:e619dfc8-9394-40de-8244-b7adc5e081aa nodeName:}" failed. No retries permitted until 2026-09-18 17:50:31.768221526 +0000 UTC m=+35.846025402 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpw8p" (UniqueName: "kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p") pod "network-check-target-dhmwn" (UID: "e619dfc8-9394-40de-8244-b7adc5e081aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 17:50:16.038773 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:16.038691    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:16.038939 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:16.038836    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:17.039148 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:17.039118    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:17.039606 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:17.039122    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:17.039606 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:17.039259    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:17.039606 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:17.039298    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:18.039903 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:18.039871    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:18.040309 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:18.039978    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:19.039043 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.038840    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:19.039126 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.038892    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:19.039170 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:19.039152    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:19.039230 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:19.039212    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:19.254580 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.254555    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 17:50:19.255035 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.254932    2568 generic.go:356] "Generic (PLEG): container finished" podID="bffad327-4f7a-40a1-b969-01a7799008f1" containerID="3df24c8843bf1da00266d97f415d79929b2428f909a97ca6085c3d66eed9098d" exitCode=1
Sep 18 17:50:19.255035 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.254962    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn" event={"ID":"bffad327-4f7a-40a1-b969-01a7799008f1","Type":"ContainerDied","Data":"3df24c8843bf1da00266d97f415d79929b2428f909a97ca6085c3d66eed9098d"}
Sep 18 17:50:19.255035 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.255036    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn" event={"ID":"bffad327-4f7a-40a1-b969-01a7799008f1","Type":"ContainerStarted","Data":"e517fcd666d6171f602508df14c903bf3dfb1bbf017e235c43f8ebdaeaef9dfe"}
Sep 18 17:50:19.257790 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.257357    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t457f" event={"ID":"85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1","Type":"ContainerStarted","Data":"1633371501ca6e65cdf544c0a7bc79475dbe451c02aa8297f2cb487b46099662"}
Sep 18 17:50:19.258887 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.258860    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rqj8n" event={"ID":"885535b4-6728-4f1f-811e-8b04785fc02d","Type":"ContainerStarted","Data":"804fb77b611a12ec922bd6f939b22f866fe0cc7f121b85eb645a9b5a811eb376"}
Sep 18 17:50:19.260652 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.260563    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nhbw2" event={"ID":"968ec864-e2d4-4577-9a8e-4e63b405acfd","Type":"ContainerStarted","Data":"f8579f854c98720932c294ceceb8aad11dae572da3cca421ae850cc7c7d19b39"}
Sep 18 17:50:19.262272 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.262249    2568 generic.go:356] "Generic (PLEG): container finished" podID="a2359522-605f-4dc9-9837-4b6f95ed99f6" containerID="5ccdd18f0aaa2ee0b49db559d74cb4e7f589c3de6dccd5c6930472f2e9ff2f49" exitCode=0
Sep 18 17:50:19.262369 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.262346    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8w95" event={"ID":"a2359522-605f-4dc9-9837-4b6f95ed99f6","Type":"ContainerDied","Data":"5ccdd18f0aaa2ee0b49db559d74cb4e7f589c3de6dccd5c6930472f2e9ff2f49"}
Sep 18 17:50:19.263854 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.263825    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n6fx7" event={"ID":"bd78b5f9-264b-477b-8deb-b91499c6a278","Type":"ContainerStarted","Data":"b37fddc3c1970b9b82c08d6974c99bba8403716f0a7217fac2fc16d94e242cf1"}
Sep 18 17:50:19.265385 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.265361    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-67z6x" event={"ID":"844bcd58-1138-4039-84dc-327f7876529d","Type":"ContainerStarted","Data":"d439901412ee3c070699b2aac2f266d3b31d976d52e0a21c0a6f07cd96a04d64"}
Sep 18 17:50:19.266832 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.266815    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh" event={"ID":"98bc124f-b527-4aee-b192-96d7f357c5b6","Type":"ContainerStarted","Data":"84aa2b55d5c36ead636b5ff45b872e131a6291c1a13dd28ffd181caeb9fd7020"}
Sep 18 17:50:19.280159 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.280111    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-66.ec2.internal" podStartSLOduration=20.28009236 podStartE2EDuration="20.28009236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 17:49:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:50:03.240287215 +0000 UTC m=+7.318091115" watchObservedRunningTime="2026-09-18 17:50:19.28009236 +0000 UTC m=+23.357896260"
Sep 18 17:50:19.308624 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.308579    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-t457f" podStartSLOduration=3.713399338 podStartE2EDuration="21.308562106s" totalImagesPullingTime="17.595162768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 17:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:50:19.279883985 +0000 UTC m=+23.357687884" watchObservedRunningTime="2026-09-18 17:50:19.308562106 +0000 UTC m=+23.386366005"
Sep 18 17:50:19.309085 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.309047    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nhbw2" podStartSLOduration=3.827160107 podStartE2EDuration="21.309032732s" totalImagesPullingTime="17.481872625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 17:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:50:19.308104664 +0000 UTC m=+23.385908564" watchObservedRunningTime="2026-09-18 17:50:19.309032732 +0000 UTC m=+23.386836632"
Sep 18 17:50:19.360713 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.360670    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-n6fx7" podStartSLOduration=3.874462952 podStartE2EDuration="21.36065472s" totalImagesPullingTime="17.486191768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:50:19.334153838 +0000 UTC m=+23.411957733" watchObservedRunningTime="2026-09-18 17:50:19.36065472 +0000 UTC m=+23.438458619"
Sep 18 17:50:19.360817 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.360772    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-67z6x" podStartSLOduration=3.751662619 podStartE2EDuration="21.360767507s" totalImagesPullingTime="17.609104888s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 17:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:50:19.360287602 +0000 UTC m=+23.438091501" watchObservedRunningTime="2026-09-18 17:50:19.360767507 +0000 UTC m=+23.438571406"
Sep 18 17:50:19.396380 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:19.396208    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rqj8n" podStartSLOduration=3.8774869990000003 podStartE2EDuration="21.396172877s" totalImagesPullingTime="17.518685878s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 17:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:50:19.396003907 +0000 UTC m=+23.473807805" watchObservedRunningTime="2026-09-18 17:50:19.396172877 +0000 UTC m=+23.473976776"
Sep 18 17:50:20.038643 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:20.038606    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:20.038867 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:20.038745    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:20.272950 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:20.272919    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 17:50:20.273468 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:20.273372    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn" event={"ID":"bffad327-4f7a-40a1-b969-01a7799008f1","Type":"ContainerStarted","Data":"8944b66ce529b1b24c5deecffc0e5cc27d4898f49b22fb6665cf9889d2a9ca04"}
Sep 18 17:50:20.273468 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:20.273445    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn" event={"ID":"bffad327-4f7a-40a1-b969-01a7799008f1","Type":"ContainerStarted","Data":"e95698d5b61d0942d582a1777ba1343473251d357a9383819df100a84503db6b"}
Sep 18 17:50:20.273581 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:20.273476    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn" event={"ID":"bffad327-4f7a-40a1-b969-01a7799008f1","Type":"ContainerStarted","Data":"2d303a5cd30076064414edbe017b0cce3266dcc9221f064f1bb67b91db826a97"}
Sep 18 17:50:20.273581 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:20.273511    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn" event={"ID":"bffad327-4f7a-40a1-b969-01a7799008f1","Type":"ContainerStarted","Data":"c05a11920ef5471c39b262a654b7bec6bda185d6ce1e6ccd7a753e23da3f1aca"}
Sep 18 17:50:20.275047 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:20.275015    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bc6dq" event={"ID":"bbd2debc-2140-487d-ac31-a1971f9e89ee","Type":"ContainerStarted","Data":"72b0153ae4f440810f2dbfd37bb5bfccea19b5c6a35f2e76536cb0b0e92b6e28"}
Sep 18 17:50:20.304986 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:20.304937    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bc6dq" podStartSLOduration=4.7398679470000005 podStartE2EDuration="22.304922281s" totalImagesPullingTime="17.565054334s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 17:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:50:20.304588876 +0000 UTC m=+24.382392775" watchObservedRunningTime="2026-09-18 17:50:20.304922281 +0000 UTC m=+24.382726179"
Sep 18 17:50:20.487687 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:20.487660    2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 17:50:21.011385 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:21.011255    2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T17:50:20.48767929Z","UUID":"bb20a39c-5cc9-4f98-b0a4-9afcd418b0b7","Handler":null,"Name":"","Endpoint":""}
Sep 18 17:50:21.013092 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:21.013055    2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 17:50:21.013092 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:21.013095    2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 17:50:21.038952 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:21.038925    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:21.039107 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:21.039048    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:21.039107 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:21.038925    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:21.039235 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:21.039169    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:21.280999 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:21.280910    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh" event={"ID":"98bc124f-b527-4aee-b192-96d7f357c5b6","Type":"ContainerStarted","Data":"7f3cd36e30fd3a2fa39aad7f8b995262159d9adbb4839a01c4045840ffeb5c82"}
Sep 18 17:50:22.042094 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:22.042028    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:22.042277 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:22.042137    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:23.038612 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:23.038562    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:23.039075 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:23.038694    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:23.039075 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:23.038750    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:23.039075 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:23.038881    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:23.129197 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:23.129150    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-n6fx7"
Sep 18 17:50:23.129795 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:23.129777    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-n6fx7"
Sep 18 17:50:24.042303 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:24.042059    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:24.042707 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:24.042330    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:24.289808 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:24.289771    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh" event={"ID":"98bc124f-b527-4aee-b192-96d7f357c5b6","Type":"ContainerStarted","Data":"cbf753dce22612cec78d8e1d6bca660f836ff5557ac796409edb231b8c7b3d48"}
Sep 18 17:50:24.292838 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:24.292770    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 17:50:24.293213 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:24.293159    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn" event={"ID":"bffad327-4f7a-40a1-b969-01a7799008f1","Type":"ContainerStarted","Data":"4daae3a16bed87754554f31e5195a7b68783544a271bcb6671f188b250809237"}
Sep 18 17:50:24.294874 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:24.294852    2568 generic.go:356] "Generic (PLEG): container finished" podID="a2359522-605f-4dc9-9837-4b6f95ed99f6" containerID="99fffcb4930711df4184505b16335411cce1cd995116b745b600c897ce06ab98" exitCode=0
Sep 18 17:50:24.294998 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:24.294880    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8w95" event={"ID":"a2359522-605f-4dc9-9837-4b6f95ed99f6","Type":"ContainerDied","Data":"99fffcb4930711df4184505b16335411cce1cd995116b745b600c897ce06ab98"}
Sep 18 17:50:24.307829 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:24.307792    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wqhmh" podStartSLOduration=4.538072954 podStartE2EDuration="26.307778375s" totalImagesPullingTime="21.769705421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 17:49:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:50:24.307762053 +0000 UTC m=+28.385565954" watchObservedRunningTime="2026-09-18 17:50:24.307778375 +0000 UTC m=+28.385582274"
Sep 18 17:50:25.039164 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:25.039126    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:25.039371 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:25.039130    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:25.039371 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:25.039271    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:25.039371 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:25.039307    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:25.502532 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:25.502508    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-n6fx7"
Sep 18 17:50:25.503048 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:25.503030    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-n6fx7"
Sep 18 17:50:26.041458 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:26.041430    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:26.041633 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:26.041520    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:26.302498 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:26.302422    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 17:50:26.302846 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:26.302822    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn" event={"ID":"bffad327-4f7a-40a1-b969-01a7799008f1","Type":"ContainerStarted","Data":"4f0358a0465b2168ea3c96f0cac89c8074dfbeedb76ece86f34e4a40e81d0890"}
Sep 18 17:50:26.303088 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:26.303068    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:26.303216 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:26.303132    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:26.303283 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:26.303268    2568 scope.go:117] "RemoveContainer" containerID="3df24c8843bf1da00266d97f415d79929b2428f909a97ca6085c3d66eed9098d"
Sep 18 17:50:26.305388 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:26.305362    2568 generic.go:356] "Generic (PLEG): container finished" podID="a2359522-605f-4dc9-9837-4b6f95ed99f6" containerID="520893768a568c8d366555b2a16c1aeca555d376ba37486c479076ed281c0dd1" exitCode=0
Sep 18 17:50:26.305494 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:26.305446    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8w95" event={"ID":"a2359522-605f-4dc9-9837-4b6f95ed99f6","Type":"ContainerDied","Data":"520893768a568c8d366555b2a16c1aeca555d376ba37486c479076ed281c0dd1"}
Sep 18 17:50:26.320078 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:26.320057    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:27.039424 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:27.039253    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:27.040003 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:27.039253    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:27.040003 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:27.039518    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:27.040003 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:27.039564    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:27.314780 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:27.314750    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8w95" event={"ID":"a2359522-605f-4dc9-9837-4b6f95ed99f6","Type":"ContainerStarted","Data":"4b619eed19c022325f183b8220c82f027edb03f141b1b8def22ab8e14b2b2980"}
Sep 18 17:50:27.319329 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:27.319306    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 17:50:27.319760 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:27.319732    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn" event={"ID":"bffad327-4f7a-40a1-b969-01a7799008f1","Type":"ContainerStarted","Data":"a232dec194052a8c3aea62b96e8abf818addb0a259def22b7483f44ee8bee007"}
Sep 18 17:50:27.320120 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:27.320096    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:27.338569 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:27.338278    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:27.351823 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:27.351765    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn" podStartSLOduration=11.757813779 podStartE2EDuration="29.351742584s" totalImagesPullingTime="17.593928805s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 17:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:50:27.35000488 +0000 UTC m=+31.427808792" watchObservedRunningTime="2026-09-18 17:50:27.351742584 +0000 UTC m=+31.429546484"
Sep 18 17:50:27.520126 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:27.520091    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dhmwn"]
Sep 18 17:50:27.520318 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:27.520245    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:27.520390 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:27.520342    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:27.520816 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:27.520788    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-66bl2"]
Sep 18 17:50:27.520971 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:27.520893    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:27.521090 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:27.520983    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:27.522276 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:27.522222    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-268np"]
Sep 18 17:50:27.522376 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:27.522360    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:27.522481 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:27.522461    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:28.325340 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:28.325303    2568 generic.go:356] "Generic (PLEG): container finished" podID="a2359522-605f-4dc9-9837-4b6f95ed99f6" containerID="4b619eed19c022325f183b8220c82f027edb03f141b1b8def22ab8e14b2b2980" exitCode=0
Sep 18 17:50:28.325737 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:28.325379    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8w95" event={"ID":"a2359522-605f-4dc9-9837-4b6f95ed99f6","Type":"ContainerDied","Data":"4b619eed19c022325f183b8220c82f027edb03f141b1b8def22ab8e14b2b2980"}
Sep 18 17:50:29.039111 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:29.039062    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:29.039111 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:29.039092    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:29.039359 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:29.039062    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:29.039359 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:29.039222    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:29.039359 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:29.039345    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:29.039466 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:29.039415    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:31.039078 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:31.039047    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:31.039492 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:31.039047    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:31.039492 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:31.039168    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-268np" podUID="71aef7ee-527e-4b8b-b17a-578530eea139"
Sep 18 17:50:31.039492 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:31.039047    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:31.039492 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:31.039258    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhmwn" podUID="e619dfc8-9394-40de-8244-b7adc5e081aa"
Sep 18 17:50:31.039492 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:31.039315    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-66bl2" podUID="a5aed5d3-d115-484b-a2f9-54a887906baa"
Sep 18 17:50:31.689044 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:31.688959    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs\") pod \"network-metrics-daemon-268np\" (UID: \"71aef7ee-527e-4b8b-b17a-578530eea139\") " pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:31.689044 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:31.689006    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:31.689396 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:31.689117    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 17:50:31.689396 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:31.689117    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 17:50:31.689396 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:31.689201    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret podName:a5aed5d3-d115-484b-a2f9-54a887906baa nodeName:}" failed. No retries permitted until 2026-09-18 17:51:03.68916646 +0000 UTC m=+67.766970340 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret") pod "global-pull-secret-syncer-66bl2" (UID: "a5aed5d3-d115-484b-a2f9-54a887906baa") : object "kube-system"/"original-pull-secret" not registered
Sep 18 17:50:31.689396 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:31.689289    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs podName:71aef7ee-527e-4b8b-b17a-578530eea139 nodeName:}" failed. No retries permitted until 2026-09-18 17:51:03.689267949 +0000 UTC m=+67.767071831 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs") pod "network-metrics-daemon-268np" (UID: "71aef7ee-527e-4b8b-b17a-578530eea139") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 17:50:31.789986 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:31.789943    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpw8p\" (UniqueName: \"kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p\") pod \"network-check-target-dhmwn\" (UID: \"e619dfc8-9394-40de-8244-b7adc5e081aa\") " pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:31.790202 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:31.790145    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 17:50:31.790202 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:31.790167    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 17:50:31.790202 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:31.790189    2568 projected.go:196] Error preparing data for projected volume kube-api-access-lpw8p for pod openshift-network-diagnostics/network-check-target-dhmwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 17:50:31.790376 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:31.790251    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p podName:e619dfc8-9394-40de-8244-b7adc5e081aa nodeName:}" failed. No retries permitted until 2026-09-18 17:51:03.790231802 +0000 UTC m=+67.868035682 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpw8p" (UniqueName: "kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p") pod "network-check-target-dhmwn" (UID: "e619dfc8-9394-40de-8244-b7adc5e081aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 17:50:31.912788 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:31.912754    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-66.ec2.internal" event="NodeReady"
Sep 18 17:50:31.912969 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:31.912916    2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 17:50:31.966634 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:31.966555    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gtq6p"]
Sep 18 17:50:32.000318 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.000277    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mkzvh"]
Sep 18 17:50:32.000504 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.000473    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:32.003274 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.003250    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 17:50:32.003469 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.003450    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s9cmf\""
Sep 18 17:50:32.003874 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.003850    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 17:50:32.016273 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.016247    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gtq6p"]
Sep 18 17:50:32.016382 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.016281    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mkzvh"]
Sep 18 17:50:32.016382 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.016379    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mkzvh"
Sep 18 17:50:32.019780 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.019752    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 17:50:32.026222 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.026161    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 17:50:32.026654 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.026627    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 17:50:32.026726 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.026715    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hhc9g\""
Sep 18 17:50:32.092581 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.092548    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96a387f5-7cde-4173-9023-721818abe42f-tmp-dir\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:32.092997 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.092589    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/96a387f5-7cde-4173-9023-721818abe42f-config-volume\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:32.092997 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.092624    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj9bs\" (UniqueName: \"kubernetes.io/projected/96a387f5-7cde-4173-9023-721818abe42f-kube-api-access-gj9bs\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:32.092997 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.092673    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njxcf\" (UniqueName: \"kubernetes.io/projected/00e9de95-8e8f-4141-8173-8521477998f8-kube-api-access-njxcf\") pod \"ingress-canary-mkzvh\" (UID: \"00e9de95-8e8f-4141-8173-8521477998f8\") " pod="openshift-ingress-canary/ingress-canary-mkzvh"
Sep 18 17:50:32.092997 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.092707    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:32.092997 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.092732    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00e9de95-8e8f-4141-8173-8521477998f8-cert\") pod \"ingress-canary-mkzvh\" (UID: \"00e9de95-8e8f-4141-8173-8521477998f8\") " pod="openshift-ingress-canary/ingress-canary-mkzvh"
Sep 18 17:50:32.193663 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.193469    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:32.193663 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:32.193646    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 17:50:32.193889 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.193686    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00e9de95-8e8f-4141-8173-8521477998f8-cert\") pod \"ingress-canary-mkzvh\" (UID: \"00e9de95-8e8f-4141-8173-8521477998f8\") " pod="openshift-ingress-canary/ingress-canary-mkzvh"
Sep 18 17:50:32.193889 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:32.193723    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls podName:96a387f5-7cde-4173-9023-721818abe42f nodeName:}" failed. No retries permitted until 2026-09-18 17:50:32.693700801 +0000 UTC m=+36.771504695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls") pod "dns-default-gtq6p" (UID: "96a387f5-7cde-4173-9023-721818abe42f") : secret "dns-default-metrics-tls" not found
Sep 18 17:50:32.193889 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.193789    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96a387f5-7cde-4173-9023-721818abe42f-tmp-dir\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:32.193889 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.193821    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/96a387f5-7cde-4173-9023-721818abe42f-config-volume\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:32.193889 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.193864    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj9bs\" (UniqueName: \"kubernetes.io/projected/96a387f5-7cde-4173-9023-721818abe42f-kube-api-access-gj9bs\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:32.194142 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.193899    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njxcf\" (UniqueName: \"kubernetes.io/projected/00e9de95-8e8f-4141-8173-8521477998f8-kube-api-access-njxcf\") pod \"ingress-canary-mkzvh\" (UID: \"00e9de95-8e8f-4141-8173-8521477998f8\") " pod="openshift-ingress-canary/ingress-canary-mkzvh"
Sep 18 17:50:32.194444 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.194419    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96a387f5-7cde-4173-9023-721818abe42f-tmp-dir\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:32.194702 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.194681    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/96a387f5-7cde-4173-9023-721818abe42f-config-volume\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:32.198337 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.198319    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00e9de95-8e8f-4141-8173-8521477998f8-cert\") pod \"ingress-canary-mkzvh\" (UID: \"00e9de95-8e8f-4141-8173-8521477998f8\") " pod="openshift-ingress-canary/ingress-canary-mkzvh"
Sep 18 17:50:32.203843 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.203728    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj9bs\" (UniqueName: \"kubernetes.io/projected/96a387f5-7cde-4173-9023-721818abe42f-kube-api-access-gj9bs\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:32.206341 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.206321    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njxcf\" (UniqueName: \"kubernetes.io/projected/00e9de95-8e8f-4141-8173-8521477998f8-kube-api-access-njxcf\") pod \"ingress-canary-mkzvh\" (UID: \"00e9de95-8e8f-4141-8173-8521477998f8\") " pod="openshift-ingress-canary/ingress-canary-mkzvh"
Sep 18 17:50:32.325246 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.325144    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mkzvh"
Sep 18 17:50:32.698674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.698639    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:32.698821 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:32.698760    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 17:50:32.698867 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:32.698825    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls podName:96a387f5-7cde-4173-9023-721818abe42f nodeName:}" failed. No retries permitted until 2026-09-18 17:50:33.698808034 +0000 UTC m=+37.776611915 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls") pod "dns-default-gtq6p" (UID: "96a387f5-7cde-4173-9023-721818abe42f") : secret "dns-default-metrics-tls" not found
Sep 18 17:50:32.866417 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:32.866379    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mkzvh"]
Sep 18 17:50:32.870680 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:50:32.870653    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00e9de95_8e8f_4141_8173_8521477998f8.slice/crio-cc22fe044ec2af1a60b2f8885b5235b56653d78e4ea402cef402081cd2f72fbd WatchSource:0}: Error finding container cc22fe044ec2af1a60b2f8885b5235b56653d78e4ea402cef402081cd2f72fbd: Status 404 returned error can't find the container with id cc22fe044ec2af1a60b2f8885b5235b56653d78e4ea402cef402081cd2f72fbd
Sep 18 17:50:33.038476 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:33.038449    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:50:33.038476 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:33.038472    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:50:33.038728 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:33.038493    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:50:33.042112 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:33.042086    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 17:50:33.042240 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:33.042112    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r5l4c\""
Sep 18 17:50:33.042240 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:33.042167    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 17:50:33.042364 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:33.042115    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 17:50:33.042364 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:33.042298    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 17:50:33.042458 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:33.042366    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sdsc5\""
Sep 18 17:50:33.341610 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:33.341573    2568 generic.go:356] "Generic (PLEG): container finished" podID="a2359522-605f-4dc9-9837-4b6f95ed99f6" containerID="2f49c54351fdbc2f35ed82e8f4a5de147a8034b35e23845370530e4113e76c99" exitCode=0
Sep 18 17:50:33.342359 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:33.341650    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8w95" event={"ID":"a2359522-605f-4dc9-9837-4b6f95ed99f6","Type":"ContainerDied","Data":"2f49c54351fdbc2f35ed82e8f4a5de147a8034b35e23845370530e4113e76c99"}
Sep 18 17:50:33.343470 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:33.343444    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mkzvh" event={"ID":"00e9de95-8e8f-4141-8173-8521477998f8","Type":"ContainerStarted","Data":"cc22fe044ec2af1a60b2f8885b5235b56653d78e4ea402cef402081cd2f72fbd"}
Sep 18 17:50:33.706914 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:33.706880    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:33.707092 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:33.707049    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 17:50:33.707158 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:33.707118    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls podName:96a387f5-7cde-4173-9023-721818abe42f nodeName:}" failed. No retries permitted until 2026-09-18 17:50:35.707099056 +0000 UTC m=+39.784902937 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls") pod "dns-default-gtq6p" (UID: "96a387f5-7cde-4173-9023-721818abe42f") : secret "dns-default-metrics-tls" not found
Sep 18 17:50:34.352894 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:34.352689    2568 generic.go:356] "Generic (PLEG): container finished" podID="a2359522-605f-4dc9-9837-4b6f95ed99f6" containerID="01073e481fb6bf6fb1f26e46a99f89ddaffd26915c7e94e71c2443aa104d9087" exitCode=0
Sep 18 17:50:34.352894 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:34.352770    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8w95" event={"ID":"a2359522-605f-4dc9-9837-4b6f95ed99f6","Type":"ContainerDied","Data":"01073e481fb6bf6fb1f26e46a99f89ddaffd26915c7e94e71c2443aa104d9087"}
Sep 18 17:50:35.356833 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:35.356791    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mkzvh" event={"ID":"00e9de95-8e8f-4141-8173-8521477998f8","Type":"ContainerStarted","Data":"a95cb95d942c1debae14eee571e656b5f3b6c1574d295bf4b8f62170d4efecd0"}
Sep 18 17:50:35.360024 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:35.359997    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v8w95" event={"ID":"a2359522-605f-4dc9-9837-4b6f95ed99f6","Type":"ContainerStarted","Data":"d24ea8b54551b35ee94a637e78935906389dc0fddabbc117ea4e23cf9bad0f0f"}
Sep 18 17:50:35.415410 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:35.415363    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mkzvh" podStartSLOduration=2.363764937 podStartE2EDuration="4.415346816s" totalImagesPullingTime="2.051581879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:50:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:50:35.375866843 +0000 UTC m=+39.453670742" watchObservedRunningTime="2026-09-18 17:50:35.415346816 +0000 UTC m=+39.493150715"
Sep 18 17:50:35.415833 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:35.415665    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-v8w95" podStartSLOduration=9.066971688 podStartE2EDuration="37.415650621s" totalImagesPullingTime="28.348678933s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 17:49:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:50:35.414486323 +0000 UTC m=+39.492290264" watchObservedRunningTime="2026-09-18 17:50:35.415650621 +0000 UTC m=+39.493454523"
Sep 18 17:50:35.721289 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:35.721251    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:35.721475 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:35.721396    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 17:50:35.721475 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:35.721457    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls podName:96a387f5-7cde-4173-9023-721818abe42f nodeName:}" failed. No retries permitted until 2026-09-18 17:50:39.721442002 +0000 UTC m=+43.799245879 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls") pod "dns-default-gtq6p" (UID: "96a387f5-7cde-4173-9023-721818abe42f") : secret "dns-default-metrics-tls" not found
Sep 18 17:50:39.751042 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:39.750990    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:39.751456 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:39.751147    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 17:50:39.751456 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:39.751231    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls podName:96a387f5-7cde-4173-9023-721818abe42f nodeName:}" failed. No retries permitted until 2026-09-18 17:50:47.751212346 +0000 UTC m=+51.829016228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls") pod "dns-default-gtq6p" (UID: "96a387f5-7cde-4173-9023-721818abe42f") : secret "dns-default-metrics-tls" not found
Sep 18 17:50:40.803056 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:40.803023    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-kq8d9"]
Sep 18 17:50:40.806015 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:40.805994    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-kq8d9"
Sep 18 17:50:40.809339 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:40.809320    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-pppdr\""
Sep 18 17:50:40.809339 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:40.809330    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 18 17:50:40.809491 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:40.809334    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 18 17:50:40.816488 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:40.816467    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-kq8d9"]
Sep 18 17:50:40.957710 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:40.957670    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qlxf\" (UniqueName: \"kubernetes.io/projected/351a37c9-f024-4310-9e05-d3d73de5985a-kube-api-access-9qlxf\") pod \"migrator-76794cbd6c-kq8d9\" (UID: \"351a37c9-f024-4310-9e05-d3d73de5985a\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-kq8d9"
Sep 18 17:50:41.058543 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.058451    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qlxf\" (UniqueName: \"kubernetes.io/projected/351a37c9-f024-4310-9e05-d3d73de5985a-kube-api-access-9qlxf\") pod \"migrator-76794cbd6c-kq8d9\" (UID: \"351a37c9-f024-4310-9e05-d3d73de5985a\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-kq8d9"
Sep 18 17:50:41.068359 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.068327    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qlxf\" (UniqueName: \"kubernetes.io/projected/351a37c9-f024-4310-9e05-d3d73de5985a-kube-api-access-9qlxf\") pod \"migrator-76794cbd6c-kq8d9\" (UID: \"351a37c9-f024-4310-9e05-d3d73de5985a\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-kq8d9"
Sep 18 17:50:41.114781 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.114747    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-kq8d9"
Sep 18 17:50:41.383989 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.383916    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-kq8d9"]
Sep 18 17:50:41.387062 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:50:41.387034    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod351a37c9_f024_4310_9e05_d3d73de5985a.slice/crio-ca9f3e8178db556079f76294a6f8cd3046592ba77a745d4f12f2d71defb5d1b4 WatchSource:0}: Error finding container ca9f3e8178db556079f76294a6f8cd3046592ba77a745d4f12f2d71defb5d1b4: Status 404 returned error can't find the container with id ca9f3e8178db556079f76294a6f8cd3046592ba77a745d4f12f2d71defb5d1b4
Sep 18 17:50:41.454160 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.454119    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qmz9s"]
Sep 18 17:50:41.458343 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.458323    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:41.461085 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.461058    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 17:50:41.461285 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.461272    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 17:50:41.461664 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.461648    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 17:50:41.461753 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.461740    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 17:50:41.464087 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.464069    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vmk8k\""
Sep 18 17:50:41.469308 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.469286    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qmz9s"]
Sep 18 17:50:41.561067 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.561023    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:41.561067 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.561073    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/113e5dce-2f88-4605-b681-086bcb92bb80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:41.561321 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.561091    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b257\" (UniqueName: \"kubernetes.io/projected/113e5dce-2f88-4605-b681-086bcb92bb80-kube-api-access-2b257\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:41.561321 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.561124    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/113e5dce-2f88-4605-b681-086bcb92bb80-crio-socket\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:41.561321 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.561154    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/113e5dce-2f88-4605-b681-086bcb92bb80-data-volume\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:41.662096 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.662061    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/113e5dce-2f88-4605-b681-086bcb92bb80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:41.662096 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.662098    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b257\" (UniqueName: \"kubernetes.io/projected/113e5dce-2f88-4605-b681-086bcb92bb80-kube-api-access-2b257\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:41.662361 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.662139    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/113e5dce-2f88-4605-b681-086bcb92bb80-crio-socket\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:41.662361 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.662157    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/113e5dce-2f88-4605-b681-086bcb92bb80-data-volume\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:41.662361 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:41.662217    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 17:50:41.662361 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.662273    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:41.662361 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:41.662307    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/113e5dce-2f88-4605-b681-086bcb92bb80-kube-rbac-proxy-cm podName:113e5dce-2f88-4605-b681-086bcb92bb80 nodeName:}" failed. No retries permitted until 2026-09-18 17:50:42.162288029 +0000 UTC m=+46.240091908 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/113e5dce-2f88-4605-b681-086bcb92bb80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qmz9s" (UID: "113e5dce-2f88-4605-b681-086bcb92bb80") : configmap "kube-rbac-proxy" not found
Sep 18 17:50:41.662361 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:41.662349    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 17:50:41.662623 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.662348    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/113e5dce-2f88-4605-b681-086bcb92bb80-crio-socket\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:41.662623 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:41.662482    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls podName:113e5dce-2f88-4605-b681-086bcb92bb80 nodeName:}" failed. No retries permitted until 2026-09-18 17:50:42.162465644 +0000 UTC m=+46.240269524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmz9s" (UID: "113e5dce-2f88-4605-b681-086bcb92bb80") : secret "insights-runtime-extractor-tls" not found
Sep 18 17:50:41.662623 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.662488    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/113e5dce-2f88-4605-b681-086bcb92bb80-data-volume\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:41.682600 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:41.682569    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b257\" (UniqueName: \"kubernetes.io/projected/113e5dce-2f88-4605-b681-086bcb92bb80-kube-api-access-2b257\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:42.170145 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:42.170104    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/113e5dce-2f88-4605-b681-086bcb92bb80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:42.170586 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:42.170244    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:42.170586 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:42.170282    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 17:50:42.170586 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:42.170355    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 17:50:42.170586 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:42.170357    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/113e5dce-2f88-4605-b681-086bcb92bb80-kube-rbac-proxy-cm podName:113e5dce-2f88-4605-b681-086bcb92bb80 nodeName:}" failed. No retries permitted until 2026-09-18 17:50:43.17033875 +0000 UTC m=+47.248142640 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/113e5dce-2f88-4605-b681-086bcb92bb80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qmz9s" (UID: "113e5dce-2f88-4605-b681-086bcb92bb80") : configmap "kube-rbac-proxy" not found
Sep 18 17:50:42.170586 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:42.170402    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls podName:113e5dce-2f88-4605-b681-086bcb92bb80 nodeName:}" failed. No retries permitted until 2026-09-18 17:50:43.170391467 +0000 UTC m=+47.248195345 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmz9s" (UID: "113e5dce-2f88-4605-b681-086bcb92bb80") : secret "insights-runtime-extractor-tls" not found
Sep 18 17:50:42.378096 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:42.378037    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-kq8d9" event={"ID":"351a37c9-f024-4310-9e05-d3d73de5985a","Type":"ContainerStarted","Data":"ca9f3e8178db556079f76294a6f8cd3046592ba77a745d4f12f2d71defb5d1b4"}
Sep 18 17:50:43.176751 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.176713    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:43.176751 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.176765    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/113e5dce-2f88-4605-b681-086bcb92bb80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:43.177266 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:43.176839    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 17:50:43.177266 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:43.176875    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 17:50:43.177266 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:43.176892    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/113e5dce-2f88-4605-b681-086bcb92bb80-kube-rbac-proxy-cm podName:113e5dce-2f88-4605-b681-086bcb92bb80 nodeName:}" failed. No retries permitted until 2026-09-18 17:50:45.1768779 +0000 UTC m=+49.254681776 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/113e5dce-2f88-4605-b681-086bcb92bb80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qmz9s" (UID: "113e5dce-2f88-4605-b681-086bcb92bb80") : configmap "kube-rbac-proxy" not found
Sep 18 17:50:43.177266 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:43.176935    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls podName:113e5dce-2f88-4605-b681-086bcb92bb80 nodeName:}" failed. No retries permitted until 2026-09-18 17:50:45.176920234 +0000 UTC m=+49.254724115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmz9s" (UID: "113e5dce-2f88-4605-b681-086bcb92bb80") : secret "insights-runtime-extractor-tls" not found
Sep 18 17:50:43.295165 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.295141    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-l8fkk"]
Sep 18 17:50:43.317311 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.317290    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-l8fkk"]
Sep 18 17:50:43.317415 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.317378    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-l8fkk"
Sep 18 17:50:43.319947 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.319926    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 18 17:50:43.320742 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.320718    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 18 17:50:43.320834 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.320722    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 18 17:50:43.320834 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.320769    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 18 17:50:43.320834 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.320752    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-v9kzf\""
Sep 18 17:50:43.382070 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.382041    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-kq8d9" event={"ID":"351a37c9-f024-4310-9e05-d3d73de5985a","Type":"ContainerStarted","Data":"f2382298221168d1ea0f2faeb6395147537410756ed70eca1f0431d3c634e59d"}
Sep 18 17:50:43.478325 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.478294    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9c5d8b14-8b39-4517-ae98-27f479acfdd8-signing-cabundle\") pod \"service-ca-55f6fc7b86-l8fkk\" (UID: \"9c5d8b14-8b39-4517-ae98-27f479acfdd8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-l8fkk"
Sep 18 17:50:43.478451 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.478364    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd8cq\" (UniqueName: \"kubernetes.io/projected/9c5d8b14-8b39-4517-ae98-27f479acfdd8-kube-api-access-nd8cq\") pod \"service-ca-55f6fc7b86-l8fkk\" (UID: \"9c5d8b14-8b39-4517-ae98-27f479acfdd8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-l8fkk"
Sep 18 17:50:43.478451 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.478392    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9c5d8b14-8b39-4517-ae98-27f479acfdd8-signing-key\") pod \"service-ca-55f6fc7b86-l8fkk\" (UID: \"9c5d8b14-8b39-4517-ae98-27f479acfdd8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-l8fkk"
Sep 18 17:50:43.578829 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.578796    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9c5d8b14-8b39-4517-ae98-27f479acfdd8-signing-cabundle\") pod \"service-ca-55f6fc7b86-l8fkk\" (UID: \"9c5d8b14-8b39-4517-ae98-27f479acfdd8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-l8fkk"
Sep 18 17:50:43.578969 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.578862    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd8cq\" (UniqueName: \"kubernetes.io/projected/9c5d8b14-8b39-4517-ae98-27f479acfdd8-kube-api-access-nd8cq\") pod \"service-ca-55f6fc7b86-l8fkk\" (UID: \"9c5d8b14-8b39-4517-ae98-27f479acfdd8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-l8fkk"
Sep 18 17:50:43.578969 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.578891    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9c5d8b14-8b39-4517-ae98-27f479acfdd8-signing-key\") pod \"service-ca-55f6fc7b86-l8fkk\" (UID: \"9c5d8b14-8b39-4517-ae98-27f479acfdd8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-l8fkk"
Sep 18 17:50:43.579510 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.579486    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9c5d8b14-8b39-4517-ae98-27f479acfdd8-signing-cabundle\") pod \"service-ca-55f6fc7b86-l8fkk\" (UID: \"9c5d8b14-8b39-4517-ae98-27f479acfdd8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-l8fkk"
Sep 18 17:50:43.581236 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.581212    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9c5d8b14-8b39-4517-ae98-27f479acfdd8-signing-key\") pod \"service-ca-55f6fc7b86-l8fkk\" (UID: \"9c5d8b14-8b39-4517-ae98-27f479acfdd8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-l8fkk"
Sep 18 17:50:43.588196 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.588162    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd8cq\" (UniqueName: \"kubernetes.io/projected/9c5d8b14-8b39-4517-ae98-27f479acfdd8-kube-api-access-nd8cq\") pod \"service-ca-55f6fc7b86-l8fkk\" (UID: \"9c5d8b14-8b39-4517-ae98-27f479acfdd8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-l8fkk"
Sep 18 17:50:43.625214 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.625187    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-l8fkk"
Sep 18 17:50:43.925314 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:43.925237    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-l8fkk"]
Sep 18 17:50:43.929267 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:50:43.929239    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c5d8b14_8b39_4517_ae98_27f479acfdd8.slice/crio-3ac99fb727011a53e18ca328d3708ad886700b52ac3b2766b9095eb438c3d2ae WatchSource:0}: Error finding container 3ac99fb727011a53e18ca328d3708ad886700b52ac3b2766b9095eb438c3d2ae: Status 404 returned error can't find the container with id 3ac99fb727011a53e18ca328d3708ad886700b52ac3b2766b9095eb438c3d2ae
Sep 18 17:50:44.385374 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:44.385343    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-l8fkk" event={"ID":"9c5d8b14-8b39-4517-ae98-27f479acfdd8","Type":"ContainerStarted","Data":"3ac99fb727011a53e18ca328d3708ad886700b52ac3b2766b9095eb438c3d2ae"}
Sep 18 17:50:44.387034 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:44.387004    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-kq8d9" event={"ID":"351a37c9-f024-4310-9e05-d3d73de5985a","Type":"ContainerStarted","Data":"f7fa58f382787b475b4b9c3ccd2eff8beaf85edf1aa604dc24b1c6740ecd1dab"}
Sep 18 17:50:44.406470 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:44.406416    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-kq8d9" podStartSLOduration=2.544132682 podStartE2EDuration="4.406399615s" totalImagesPullingTime="1.862266933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:50:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:50:44.405384497 +0000 UTC m=+48.483188397" watchObservedRunningTime="2026-09-18 17:50:44.406399615 +0000 UTC m=+48.484203513"
Sep 18 17:50:44.408114 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:44.408097    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t457f_85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1/dns-node-resolver/0.log"
Sep 18 17:50:45.193068 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:45.193028    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:45.193290 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:45.193095    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/113e5dce-2f88-4605-b681-086bcb92bb80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:45.193290 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:45.193212    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 17:50:45.193290 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:45.193246    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 17:50:45.193290 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:45.193278    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls podName:113e5dce-2f88-4605-b681-086bcb92bb80 nodeName:}" failed. No retries permitted until 2026-09-18 17:50:49.193263612 +0000 UTC m=+53.271067489 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmz9s" (UID: "113e5dce-2f88-4605-b681-086bcb92bb80") : secret "insights-runtime-extractor-tls" not found
Sep 18 17:50:45.193452 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:45.193301    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/113e5dce-2f88-4605-b681-086bcb92bb80-kube-rbac-proxy-cm podName:113e5dce-2f88-4605-b681-086bcb92bb80 nodeName:}" failed. No retries permitted until 2026-09-18 17:50:49.193285297 +0000 UTC m=+53.271089196 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/113e5dce-2f88-4605-b681-086bcb92bb80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qmz9s" (UID: "113e5dce-2f88-4605-b681-086bcb92bb80") : configmap "kube-rbac-proxy" not found
Sep 18 17:50:45.208866 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:45.208842    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nhbw2_968ec864-e2d4-4577-9a8e-4e63b405acfd/node-ca/0.log"
Sep 18 17:50:46.215253 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:46.215225    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mkzvh_00e9de95-8e8f-4141-8173-8521477998f8/serve-healthcheck-canary/0.log"
Sep 18 17:50:46.393943 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:46.393866    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-l8fkk" event={"ID":"9c5d8b14-8b39-4517-ae98-27f479acfdd8","Type":"ContainerStarted","Data":"bd70310d92ed1a66bafcfe09568b6b9ab085d19363564d5b21fd3f8495c89ed9"}
Sep 18 17:50:46.419056 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:46.419022    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-kq8d9_351a37c9-f024-4310-9e05-d3d73de5985a/migrator/0.log"
Sep 18 17:50:46.438914 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:46.438859    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-l8fkk" podStartSLOduration=1.3241651509999999 podStartE2EDuration="3.438843526s" totalImagesPullingTime="2.114678375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:50:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:50:46.438792708 +0000 UTC m=+50.516596607" watchObservedRunningTime="2026-09-18 17:50:46.438843526 +0000 UTC m=+50.516647424"
Sep 18 17:50:46.611138 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:46.611109    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-kq8d9_351a37c9-f024-4310-9e05-d3d73de5985a/graceful-termination/0.log"
Sep 18 17:50:47.816770 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:47.816729    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:50:47.817234 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:47.816910    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 17:50:47.817234 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:47.816992    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls podName:96a387f5-7cde-4173-9023-721818abe42f nodeName:}" failed. No retries permitted until 2026-09-18 17:51:03.81697206 +0000 UTC m=+67.894775952 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls") pod "dns-default-gtq6p" (UID: "96a387f5-7cde-4173-9023-721818abe42f") : secret "dns-default-metrics-tls" not found
Sep 18 17:50:49.227499 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:49.227457    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/113e5dce-2f88-4605-b681-086bcb92bb80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:49.227916 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:49.227562    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:49.227916 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:49.227668    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 17:50:49.227916 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:50:49.227723    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls podName:113e5dce-2f88-4605-b681-086bcb92bb80 nodeName:}" failed. No retries permitted until 2026-09-18 17:50:57.227707552 +0000 UTC m=+61.305511447 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qmz9s" (UID: "113e5dce-2f88-4605-b681-086bcb92bb80") : secret "insights-runtime-extractor-tls" not found
Sep 18 17:50:49.228027 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:49.227964    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/113e5dce-2f88-4605-b681-086bcb92bb80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:57.287743 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:57.287696    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:57.290080 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:57.290050    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/113e5dce-2f88-4605-b681-086bcb92bb80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qmz9s\" (UID: \"113e5dce-2f88-4605-b681-086bcb92bb80\") " pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:57.368078 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:57.368045    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qmz9s"
Sep 18 17:50:57.659968 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:57.659944    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qmz9s"]
Sep 18 17:50:57.662102 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:50:57.662070    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod113e5dce_2f88_4605_b681_086bcb92bb80.slice/crio-4cb9fe5287212ed8314192d3ba9f9a3ba65050af2237f8693aff071e76b36cc9 WatchSource:0}: Error finding container 4cb9fe5287212ed8314192d3ba9f9a3ba65050af2237f8693aff071e76b36cc9: Status 404 returned error can't find the container with id 4cb9fe5287212ed8314192d3ba9f9a3ba65050af2237f8693aff071e76b36cc9
Sep 18 17:50:58.425925 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:58.425889    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qmz9s" event={"ID":"113e5dce-2f88-4605-b681-086bcb92bb80","Type":"ContainerStarted","Data":"1db877ccd589d44f1672465f6f0cfe195270c50602f302044642d3be6118e9aa"}
Sep 18 17:50:58.426324 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:58.425954    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qmz9s" event={"ID":"113e5dce-2f88-4605-b681-086bcb92bb80","Type":"ContainerStarted","Data":"4cb9fe5287212ed8314192d3ba9f9a3ba65050af2237f8693aff071e76b36cc9"}
Sep 18 17:50:59.342088 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:59.342056    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-56mpn"
Sep 18 17:50:59.431276 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:50:59.431233    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qmz9s" event={"ID":"113e5dce-2f88-4605-b681-086bcb92bb80","Type":"ContainerStarted","Data":"8713899ee0bd606028b504068f2bdbdce9fbfa96fb5fad147e5c14adc7b0a96d"}
Sep 18 17:51:00.437150 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:00.437116    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qmz9s" event={"ID":"113e5dce-2f88-4605-b681-086bcb92bb80","Type":"ContainerStarted","Data":"94944d7a89eb81174860bb9e5c5be8e6234b3f4d2569204393aefc867fccd479"}
Sep 18 17:51:00.464826 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:00.464774    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qmz9s" podStartSLOduration=17.469292343 podStartE2EDuration="19.464757311s" totalImagesPullingTime="1.995464968s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 17:50:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:51:00.463916118 +0000 UTC m=+64.541720017" watchObservedRunningTime="2026-09-18 17:51:00.464757311 +0000 UTC m=+64.542561209"
Sep 18 17:51:03.733749 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.733689    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs\") pod \"network-metrics-daemon-268np\" (UID: \"71aef7ee-527e-4b8b-b17a-578530eea139\") " pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:51:03.733749 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.733757    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:51:03.736259 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.736240    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 17:51:03.736620 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.736605    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 17:51:03.746767 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.746735    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71aef7ee-527e-4b8b-b17a-578530eea139-metrics-certs\") pod \"network-metrics-daemon-268np\" (UID: \"71aef7ee-527e-4b8b-b17a-578530eea139\") " pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:51:03.746767 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.746748    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5aed5d3-d115-484b-a2f9-54a887906baa-original-pull-secret\") pod \"global-pull-secret-syncer-66bl2\" (UID: \"a5aed5d3-d115-484b-a2f9-54a887906baa\") " pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:51:03.834864 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.834825    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:51:03.834864 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.834871    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpw8p\" (UniqueName: \"kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p\") pod \"network-check-target-dhmwn\" (UID: \"e619dfc8-9394-40de-8244-b7adc5e081aa\") " pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:51:03.837900 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.837864    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 17:51:03.840037 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.840002    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96a387f5-7cde-4173-9023-721818abe42f-metrics-tls\") pod \"dns-default-gtq6p\" (UID: \"96a387f5-7cde-4173-9023-721818abe42f\") " pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:51:03.847756 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.847732    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 17:51:03.858244 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.858214    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpw8p\" (UniqueName: \"kubernetes.io/projected/e619dfc8-9394-40de-8244-b7adc5e081aa-kube-api-access-lpw8p\") pod \"network-check-target-dhmwn\" (UID: \"e619dfc8-9394-40de-8244-b7adc5e081aa\") " pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:51:03.968232 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.968200    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sdsc5\""
Sep 18 17:51:03.971073 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.971058    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-66bl2"
Sep 18 17:51:03.976222 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.976203    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-268np"
Sep 18 17:51:03.978740 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.978723    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r5l4c\""
Sep 18 17:51:03.986850 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:03.986820    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:51:04.115296 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.115263    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s9cmf\""
Sep 18 17:51:04.123454 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.123432    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:51:04.269711 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.269657    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-66bl2"]
Sep 18 17:51:04.272063 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:51:04.272036    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5aed5d3_d115_484b_a2f9_54a887906baa.slice/crio-a30e200e356528150359259f70c7e0803d917e8b0d1b9f7fc36f2096cacebabf WatchSource:0}: Error finding container a30e200e356528150359259f70c7e0803d917e8b0d1b9f7fc36f2096cacebabf: Status 404 returned error can't find the container with id a30e200e356528150359259f70c7e0803d917e8b0d1b9f7fc36f2096cacebabf
Sep 18 17:51:04.277456 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.277432    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-268np"]
Sep 18 17:51:04.293605 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.293578    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dhmwn"]
Sep 18 17:51:04.296103 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:51:04.296078    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71aef7ee_527e_4b8b_b17a_578530eea139.slice/crio-439c23f0dcf251be51bf3b2cab8fe70a9fc5f0a85a50bcfc1b6bfc019d32e051 WatchSource:0}: Error finding container 439c23f0dcf251be51bf3b2cab8fe70a9fc5f0a85a50bcfc1b6bfc019d32e051: Status 404 returned error can't find the container with id 439c23f0dcf251be51bf3b2cab8fe70a9fc5f0a85a50bcfc1b6bfc019d32e051
Sep 18 17:51:04.296796 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:51:04.296776    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode619dfc8_9394_40de_8244_b7adc5e081aa.slice/crio-57917d6b08ac686606dacd8435e3fe4786062f05e345666970db5ebf0102a470 WatchSource:0}: Error finding container 57917d6b08ac686606dacd8435e3fe4786062f05e345666970db5ebf0102a470: Status 404 returned error can't find the container with id 57917d6b08ac686606dacd8435e3fe4786062f05e345666970db5ebf0102a470
Sep 18 17:51:04.393375 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.393339    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gtq6p"]
Sep 18 17:51:04.396488 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:51:04.396462    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a387f5_7cde_4173_9023_721818abe42f.slice/crio-ea7ba8b082fb1d4fe6634d5555fae3c56825880671a39f29739080d456a01333 WatchSource:0}: Error finding container ea7ba8b082fb1d4fe6634d5555fae3c56825880671a39f29739080d456a01333: Status 404 returned error can't find the container with id ea7ba8b082fb1d4fe6634d5555fae3c56825880671a39f29739080d456a01333
Sep 18 17:51:04.452986 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.452947    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dhmwn" event={"ID":"e619dfc8-9394-40de-8244-b7adc5e081aa","Type":"ContainerStarted","Data":"57917d6b08ac686606dacd8435e3fe4786062f05e345666970db5ebf0102a470"}
Sep 18 17:51:04.454135 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.454096    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gtq6p" event={"ID":"96a387f5-7cde-4173-9023-721818abe42f","Type":"ContainerStarted","Data":"ea7ba8b082fb1d4fe6634d5555fae3c56825880671a39f29739080d456a01333"}
Sep 18 17:51:04.455232 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.455189    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-268np" event={"ID":"71aef7ee-527e-4b8b-b17a-578530eea139","Type":"ContainerStarted","Data":"439c23f0dcf251be51bf3b2cab8fe70a9fc5f0a85a50bcfc1b6bfc019d32e051"}
Sep 18 17:51:04.456201 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.456168    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-66bl2" event={"ID":"a5aed5d3-d115-484b-a2f9-54a887906baa","Type":"ContainerStarted","Data":"a30e200e356528150359259f70c7e0803d917e8b0d1b9f7fc36f2096cacebabf"}
Sep 18 17:51:04.653293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.653217    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"]
Sep 18 17:51:04.655609 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.655594    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"
Sep 18 17:51:04.657998 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.657977    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 18 17:51:04.658409 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.658387    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 18 17:51:04.658935 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.658920    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 18 17:51:04.659001 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.658945    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 18 17:51:04.677765 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.677725    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"]
Sep 18 17:51:04.742415 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.742371    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x7w6\" (UniqueName: \"kubernetes.io/projected/18a2a61a-328d-48e5-abab-f2c52ff8ea4a-kube-api-access-4x7w6\") pod \"klusterlet-addon-workmgr-5f88485c7c-55psl\" (UID: \"18a2a61a-328d-48e5-abab-f2c52ff8ea4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"
Sep 18 17:51:04.742415 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.742429    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18a2a61a-328d-48e5-abab-f2c52ff8ea4a-tmp\") pod \"klusterlet-addon-workmgr-5f88485c7c-55psl\" (UID: \"18a2a61a-328d-48e5-abab-f2c52ff8ea4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"
Sep 18 17:51:04.742925 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.742455    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/18a2a61a-328d-48e5-abab-f2c52ff8ea4a-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f88485c7c-55psl\" (UID: \"18a2a61a-328d-48e5-abab-f2c52ff8ea4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"
Sep 18 17:51:04.769658 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.769626    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9568558f4-kz48x"]
Sep 18 17:51:04.771777 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.771752    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-sj7s5"]
Sep 18 17:51:04.771934 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.771908    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.773677 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.773651    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-sj7s5"
Sep 18 17:51:04.775127 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.775104    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xssnq\""
Sep 18 17:51:04.780710 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.780686    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 18 17:51:04.781486 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.780997    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 18 17:51:04.781486 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.781171    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 17:51:04.781486 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.781371    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 18 17:51:04.781728 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.781501    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 17:51:04.781867 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.781843    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wtcfv\""
Sep 18 17:51:04.785633 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.785605    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 18 17:51:04.790917 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.790893    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-sj7s5"]
Sep 18 17:51:04.842854 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.842817    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46939250-033d-42f7-a8b8-1330f875a69b-installation-pull-secrets\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.843034 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.842871    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x7w6\" (UniqueName: \"kubernetes.io/projected/18a2a61a-328d-48e5-abab-f2c52ff8ea4a-kube-api-access-4x7w6\") pod \"klusterlet-addon-workmgr-5f88485c7c-55psl\" (UID: \"18a2a61a-328d-48e5-abab-f2c52ff8ea4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"
Sep 18 17:51:04.843034 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.842904    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46939250-033d-42f7-a8b8-1330f875a69b-bound-sa-token\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.843034 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.842928    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md75k\" (UniqueName: \"kubernetes.io/projected/408c606b-e5cc-4afc-9a3d-afcab63ba455-kube-api-access-md75k\") pod \"downloads-6594f4c75-sj7s5\" (UID: \"408c606b-e5cc-4afc-9a3d-afcab63ba455\") " pod="openshift-console/downloads-6594f4c75-sj7s5"
Sep 18 17:51:04.843034 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.842968    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46939250-033d-42f7-a8b8-1330f875a69b-registry-tls\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.843034 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.842994    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18a2a61a-328d-48e5-abab-f2c52ff8ea4a-tmp\") pod \"klusterlet-addon-workmgr-5f88485c7c-55psl\" (UID: \"18a2a61a-328d-48e5-abab-f2c52ff8ea4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"
Sep 18 17:51:04.843034 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.843025    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/18a2a61a-328d-48e5-abab-f2c52ff8ea4a-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f88485c7c-55psl\" (UID: \"18a2a61a-328d-48e5-abab-f2c52ff8ea4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"
Sep 18 17:51:04.843386 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.843057    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46939250-033d-42f7-a8b8-1330f875a69b-image-registry-private-configuration\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.843386 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.843082    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46939250-033d-42f7-a8b8-1330f875a69b-registry-certificates\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.843386 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.843115    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46939250-033d-42f7-a8b8-1330f875a69b-ca-trust-extracted\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.843386 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.843137    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46939250-033d-42f7-a8b8-1330f875a69b-trusted-ca\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.843386 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.843159    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg92d\" (UniqueName: \"kubernetes.io/projected/46939250-033d-42f7-a8b8-1330f875a69b-kube-api-access-tg92d\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.843946 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.843920    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18a2a61a-328d-48e5-abab-f2c52ff8ea4a-tmp\") pod \"klusterlet-addon-workmgr-5f88485c7c-55psl\" (UID: \"18a2a61a-328d-48e5-abab-f2c52ff8ea4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"
Sep 18 17:51:04.848898 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.848837    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/18a2a61a-328d-48e5-abab-f2c52ff8ea4a-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f88485c7c-55psl\" (UID: \"18a2a61a-328d-48e5-abab-f2c52ff8ea4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"
Sep 18 17:51:04.856646 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.856620    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x7w6\" (UniqueName: \"kubernetes.io/projected/18a2a61a-328d-48e5-abab-f2c52ff8ea4a-kube-api-access-4x7w6\") pod \"klusterlet-addon-workmgr-5f88485c7c-55psl\" (UID: \"18a2a61a-328d-48e5-abab-f2c52ff8ea4a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"
Sep 18 17:51:04.866358 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.866327    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9568558f4-kz48x"]
Sep 18 17:51:04.944429 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.944392    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46939250-033d-42f7-a8b8-1330f875a69b-installation-pull-secrets\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.944617 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.944460    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46939250-033d-42f7-a8b8-1330f875a69b-bound-sa-token\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.944617 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.944489    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md75k\" (UniqueName: \"kubernetes.io/projected/408c606b-e5cc-4afc-9a3d-afcab63ba455-kube-api-access-md75k\") pod \"downloads-6594f4c75-sj7s5\" (UID: \"408c606b-e5cc-4afc-9a3d-afcab63ba455\") " pod="openshift-console/downloads-6594f4c75-sj7s5"
Sep 18 17:51:04.944617 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.944531    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46939250-033d-42f7-a8b8-1330f875a69b-registry-tls\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.944617 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.944564    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46939250-033d-42f7-a8b8-1330f875a69b-image-registry-private-configuration\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.944617 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.944601    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46939250-033d-42f7-a8b8-1330f875a69b-registry-certificates\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.944889 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.944635    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46939250-033d-42f7-a8b8-1330f875a69b-ca-trust-extracted\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.944889 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.944658    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46939250-033d-42f7-a8b8-1330f875a69b-trusted-ca\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.944889 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.944680    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg92d\" (UniqueName: \"kubernetes.io/projected/46939250-033d-42f7-a8b8-1330f875a69b-kube-api-access-tg92d\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.946028 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.945976    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/46939250-033d-42f7-a8b8-1330f875a69b-registry-certificates\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.946229 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.946207    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/46939250-033d-42f7-a8b8-1330f875a69b-ca-trust-extracted\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.947022 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.946989    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46939250-033d-42f7-a8b8-1330f875a69b-trusted-ca\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.950368 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.950323    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/46939250-033d-42f7-a8b8-1330f875a69b-installation-pull-secrets\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.950644 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.950621    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/46939250-033d-42f7-a8b8-1330f875a69b-image-registry-private-configuration\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.951961 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.951921    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/46939250-033d-42f7-a8b8-1330f875a69b-registry-tls\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.964675 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.964572    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"
Sep 18 17:51:04.967654 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.967623    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/46939250-033d-42f7-a8b8-1330f875a69b-bound-sa-token\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.968208 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.968171    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg92d\" (UniqueName: \"kubernetes.io/projected/46939250-033d-42f7-a8b8-1330f875a69b-kube-api-access-tg92d\") pod \"image-registry-9568558f4-kz48x\" (UID: \"46939250-033d-42f7-a8b8-1330f875a69b\") " pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:04.969559 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:04.969536    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-md75k\" (UniqueName: \"kubernetes.io/projected/408c606b-e5cc-4afc-9a3d-afcab63ba455-kube-api-access-md75k\") pod \"downloads-6594f4c75-sj7s5\" (UID: \"408c606b-e5cc-4afc-9a3d-afcab63ba455\") " pod="openshift-console/downloads-6594f4c75-sj7s5"
Sep 18 17:51:05.084732 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:05.084495    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:05.092961 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:05.092930    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-sj7s5"
Sep 18 17:51:05.299707 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:05.298801    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"]
Sep 18 17:51:05.454810 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:51:05.454767    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a2a61a_328d_48e5_abab_f2c52ff8ea4a.slice/crio-6dad7160aae40d9cd77dc8cbbc407dae4349933378e28074e6ed956d65147c6b WatchSource:0}: Error finding container 6dad7160aae40d9cd77dc8cbbc407dae4349933378e28074e6ed956d65147c6b: Status 404 returned error can't find the container with id 6dad7160aae40d9cd77dc8cbbc407dae4349933378e28074e6ed956d65147c6b
Sep 18 17:51:05.467948 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:05.467884    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl" event={"ID":"18a2a61a-328d-48e5-abab-f2c52ff8ea4a","Type":"ContainerStarted","Data":"6dad7160aae40d9cd77dc8cbbc407dae4349933378e28074e6ed956d65147c6b"}
Sep 18 17:51:05.758106 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:05.757945    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-sj7s5"]
Sep 18 17:51:05.759959 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:05.759770    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9568558f4-kz48x"]
Sep 18 17:51:05.765332 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:51:05.765292    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46939250_033d_42f7_a8b8_1330f875a69b.slice/crio-d0399b9075a78fc98ab25d5f38ff4356e18cfdac1009de2624493111b782755e WatchSource:0}: Error finding container d0399b9075a78fc98ab25d5f38ff4356e18cfdac1009de2624493111b782755e: Status 404 returned error can't find the container with id d0399b9075a78fc98ab25d5f38ff4356e18cfdac1009de2624493111b782755e
Sep 18 17:51:05.765862 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:51:05.765823    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod408c606b_e5cc_4afc_9a3d_afcab63ba455.slice/crio-fa2882a3362453e0278f8f4c37723cddee58c47c220dc91dc1793958046c3101 WatchSource:0}: Error finding container fa2882a3362453e0278f8f4c37723cddee58c47c220dc91dc1793958046c3101: Status 404 returned error can't find the container with id fa2882a3362453e0278f8f4c37723cddee58c47c220dc91dc1793958046c3101
Sep 18 17:51:06.475343 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:06.475245    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-268np" event={"ID":"71aef7ee-527e-4b8b-b17a-578530eea139","Type":"ContainerStarted","Data":"4b74a47959d318ddb73bf47cfa2b480f676b9f23fbec37c39b2a3105cf9f7571"}
Sep 18 17:51:06.475343 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:06.475307    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-268np" event={"ID":"71aef7ee-527e-4b8b-b17a-578530eea139","Type":"ContainerStarted","Data":"63af8c8cab4099be7f6ff9af2820169afbad3587d4c8ca73997ed0506db55bdf"}
Sep 18 17:51:06.477751 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:06.477721    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-sj7s5" event={"ID":"408c606b-e5cc-4afc-9a3d-afcab63ba455","Type":"ContainerStarted","Data":"fa2882a3362453e0278f8f4c37723cddee58c47c220dc91dc1793958046c3101"}
Sep 18 17:51:06.480077 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:06.480042    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9568558f4-kz48x" event={"ID":"46939250-033d-42f7-a8b8-1330f875a69b","Type":"ContainerStarted","Data":"4c88b1846eee43e695133517710e3cd2409bd0fa00cade15b2ad0e881543daa0"}
Sep 18 17:51:06.480215 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:06.480090    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9568558f4-kz48x" event={"ID":"46939250-033d-42f7-a8b8-1330f875a69b","Type":"ContainerStarted","Data":"d0399b9075a78fc98ab25d5f38ff4356e18cfdac1009de2624493111b782755e"}
Sep 18 17:51:06.480276 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:06.480226    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:06.495486 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:06.495259    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-268np" podStartSLOduration=67.280623478 podStartE2EDuration="1m8.495236885s" totalImagesPullingTime="1.214613407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:51:06.493767817 +0000 UTC m=+70.571571717" watchObservedRunningTime="2026-09-18 17:51:06.495236885 +0000 UTC m=+70.573040788"
Sep 18 17:51:06.516822 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:06.516766    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9568558f4-kz48x" podStartSLOduration=2.516745508 podStartE2EDuration="2.516745508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:51:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:51:06.515833301 +0000 UTC m=+70.593637201" watchObservedRunningTime="2026-09-18 17:51:06.516745508 +0000 UTC m=+70.594549408"
Sep 18 17:51:07.489219 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:07.489161    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gtq6p" event={"ID":"96a387f5-7cde-4173-9023-721818abe42f","Type":"ContainerStarted","Data":"7c6a1430aad0cf81d39aab5dd8c3c41059708513702f709d8378b4bb06568929"}
Sep 18 17:51:08.011150 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.010055    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57d6b6fd-d5sdt"]
Sep 18 17:51:08.013521 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.013494    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.020461 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.019007    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 18 17:51:08.020461 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.019649    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 18 17:51:08.020461 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.019871    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5j4br\""
Sep 18 17:51:08.020461 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.020093    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 18 17:51:08.020461 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.020264    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 18 17:51:08.023318 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.022386    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 18 17:51:08.034132 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.034097    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57d6b6fd-d5sdt"]
Sep 18 17:51:08.071576 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.071520    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-oauth-serving-cert\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.073843 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.073255    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qvpm\" (UniqueName: \"kubernetes.io/projected/bc594e74-714a-4959-95c6-cb8a6d9979cb-kube-api-access-2qvpm\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.073843 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.073341    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-service-ca\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.073843 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.073402    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-serving-cert\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.073843 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.073478    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-oauth-config\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.073843 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.073542    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-config\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.174950 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.174907    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-oauth-serving-cert\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.175121 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.174957    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qvpm\" (UniqueName: \"kubernetes.io/projected/bc594e74-714a-4959-95c6-cb8a6d9979cb-kube-api-access-2qvpm\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.175121 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.174995    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-service-ca\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.175121 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.175032    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-serving-cert\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.175121 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.175079    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-oauth-config\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.175121 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.175112    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-config\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.175856 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.175779    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-oauth-serving-cert\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.176451 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.176404    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-service-ca\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.176547 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.176508    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-config\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.178172 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.178107    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-oauth-config\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.178305 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.178281    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-serving-cert\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.193133 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.193101    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qvpm\" (UniqueName: \"kubernetes.io/projected/bc594e74-714a-4959-95c6-cb8a6d9979cb-kube-api-access-2qvpm\") pod \"console-57d6b6fd-d5sdt\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") " pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:08.327784 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:08.327701    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:11.322561 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:11.322477    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57d6b6fd-d5sdt"]
Sep 18 17:51:11.326778 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:51:11.326747    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc594e74_714a_4959_95c6_cb8a6d9979cb.slice/crio-70edb6a614a606809356862b53f595781682710924528695069dbe9389e4cabe WatchSource:0}: Error finding container 70edb6a614a606809356862b53f595781682710924528695069dbe9389e4cabe: Status 404 returned error can't find the container with id 70edb6a614a606809356862b53f595781682710924528695069dbe9389e4cabe
Sep 18 17:51:11.508424 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:11.508387    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-66bl2" event={"ID":"a5aed5d3-d115-484b-a2f9-54a887906baa","Type":"ContainerStarted","Data":"bfd54862c57c119a490953f766551f2a60925b476b0a8becaf24757cdbcf5b0c"}
Sep 18 17:51:11.510866 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:11.510833    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dhmwn" event={"ID":"e619dfc8-9394-40de-8244-b7adc5e081aa","Type":"ContainerStarted","Data":"6f532fa84f4eea9b0b561b85dc4bfaac95f37774abf968b9ba108680387e6e45"}
Sep 18 17:51:11.510988 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:11.510893    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:51:11.512781 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:11.512742    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d6b6fd-d5sdt" event={"ID":"bc594e74-714a-4959-95c6-cb8a6d9979cb","Type":"ContainerStarted","Data":"70edb6a614a606809356862b53f595781682710924528695069dbe9389e4cabe"}
Sep 18 17:51:11.515112 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:11.515088    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gtq6p" event={"ID":"96a387f5-7cde-4173-9023-721818abe42f","Type":"ContainerStarted","Data":"0e81bc6d5091440a40f41673154ee49c4a1b172ca9af93a7fd45bf2cb86af00c"}
Sep 18 17:51:11.515325 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:11.515305    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:51:11.538003 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:11.537938    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-66bl2" podStartSLOduration=66.777426233 podStartE2EDuration="1m13.537916179s" totalImagesPullingTime="6.760489946s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 17:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:51:11.537882138 +0000 UTC m=+75.615686038" watchObservedRunningTime="2026-09-18 17:51:11.537916179 +0000 UTC m=+75.615720079"
Sep 18 17:51:11.563410 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:11.563350    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dhmwn" podStartSLOduration=66.838933435 podStartE2EDuration="1m13.563328041s" totalImagesPullingTime="6.724394606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:51:11.562396002 +0000 UTC m=+75.640199913" watchObservedRunningTime="2026-09-18 17:51:11.563328041 +0000 UTC m=+75.641131941"
Sep 18 17:51:11.607856 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:11.607763    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gtq6p" podStartSLOduration=38.335292474 podStartE2EDuration="40.607746923s" totalImagesPullingTime="2.272454449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:50:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:51:11.606945558 +0000 UTC m=+75.684749457" watchObservedRunningTime="2026-09-18 17:51:11.607746923 +0000 UTC m=+75.685550822"
Sep 18 17:51:12.100948 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.098000    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"]
Sep 18 17:51:12.103587 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.103556    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:12.109130 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.109046    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 18 17:51:12.109689 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.109656    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 17:51:12.109689 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.109682    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 17:51:12.109954 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.109935    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 17:51:12.110024 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.109995    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 18 17:51:12.110290 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.110271    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dpltc\""
Sep 18 17:51:12.119627 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.119603    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"]
Sep 18 17:51:12.207512 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.207476    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx2wv\" (UniqueName: \"kubernetes.io/projected/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-kube-api-access-cx2wv\") pod \"prometheus-operator-7bd5d56d6-hs89z\" (UID: \"7bafd0b2-409e-46ba-a0b8-9391a248ab9f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:12.207744 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.207533    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-hs89z\" (UID: \"7bafd0b2-409e-46ba-a0b8-9391a248ab9f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:12.207744 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.207557    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-hs89z\" (UID: \"7bafd0b2-409e-46ba-a0b8-9391a248ab9f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:12.207744 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.207648    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-hs89z\" (UID: \"7bafd0b2-409e-46ba-a0b8-9391a248ab9f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:12.309580 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.308957    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx2wv\" (UniqueName: \"kubernetes.io/projected/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-kube-api-access-cx2wv\") pod \"prometheus-operator-7bd5d56d6-hs89z\" (UID: \"7bafd0b2-409e-46ba-a0b8-9391a248ab9f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:12.309580 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.309011    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-hs89z\" (UID: \"7bafd0b2-409e-46ba-a0b8-9391a248ab9f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:12.309580 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.309050    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-hs89z\" (UID: \"7bafd0b2-409e-46ba-a0b8-9391a248ab9f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:12.309580 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.309121    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-hs89z\" (UID: \"7bafd0b2-409e-46ba-a0b8-9391a248ab9f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:12.309580 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:51:12.309298    2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 18 17:51:12.309580 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:51:12.309445    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-prometheus-operator-tls podName:7bafd0b2-409e-46ba-a0b8-9391a248ab9f nodeName:}" failed. No retries permitted until 2026-09-18 17:51:12.809423677 +0000 UTC m=+76.887227570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-prometheus-operator-tls") pod "prometheus-operator-7bd5d56d6-hs89z" (UID: "7bafd0b2-409e-46ba-a0b8-9391a248ab9f") : secret "prometheus-operator-tls" not found
Sep 18 17:51:12.310092 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.310067    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-hs89z\" (UID: \"7bafd0b2-409e-46ba-a0b8-9391a248ab9f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:12.312445 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.312421    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-hs89z\" (UID: \"7bafd0b2-409e-46ba-a0b8-9391a248ab9f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:12.323975 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.323953    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx2wv\" (UniqueName: \"kubernetes.io/projected/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-kube-api-access-cx2wv\") pod \"prometheus-operator-7bd5d56d6-hs89z\" (UID: \"7bafd0b2-409e-46ba-a0b8-9391a248ab9f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:12.813864 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:12.813213    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-hs89z\" (UID: \"7bafd0b2-409e-46ba-a0b8-9391a248ab9f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:12.813864 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:51:12.813418    2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 18 17:51:12.813864 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:51:12.813493    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-prometheus-operator-tls podName:7bafd0b2-409e-46ba-a0b8-9391a248ab9f nodeName:}" failed. No retries permitted until 2026-09-18 17:51:13.813474218 +0000 UTC m=+77.891278101 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-prometheus-operator-tls") pod "prometheus-operator-7bd5d56d6-hs89z" (UID: "7bafd0b2-409e-46ba-a0b8-9391a248ab9f") : secret "prometheus-operator-tls" not found
Sep 18 17:51:13.542305 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:13.542220    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl" event={"ID":"18a2a61a-328d-48e5-abab-f2c52ff8ea4a","Type":"ContainerStarted","Data":"7efdcab1d9b998d5911b4360d601e167dddcd45bae2f39a31cbdd6ef3bd90199"}
Sep 18 17:51:13.543769 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:13.543743    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"
Sep 18 17:51:13.545516 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:13.545494    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl"
Sep 18 17:51:13.563763 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:13.563700    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f88485c7c-55psl" podStartSLOduration=1.732757176 podStartE2EDuration="9.563677683s" totalImagesPullingTime="7.830920507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:51:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:51:13.562128899 +0000 UTC m=+77.639932792" watchObservedRunningTime="2026-09-18 17:51:13.563677683 +0000 UTC m=+77.641481585"
Sep 18 17:51:13.822209 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:13.822102    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-hs89z\" (UID: \"7bafd0b2-409e-46ba-a0b8-9391a248ab9f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:13.827929 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:13.827851    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bafd0b2-409e-46ba-a0b8-9391a248ab9f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-hs89z\" (UID: \"7bafd0b2-409e-46ba-a0b8-9391a248ab9f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:13.916062 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:13.916021    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"
Sep 18 17:51:14.225278 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:14.225239    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z"]
Sep 18 17:51:14.231200 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:51:14.230535    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bafd0b2_409e_46ba_a0b8_9391a248ab9f.slice/crio-e99da8bf46db1cfe522607c169b1e83e969f15bc639f328518549e6b49008359 WatchSource:0}: Error finding container e99da8bf46db1cfe522607c169b1e83e969f15bc639f328518549e6b49008359: Status 404 returned error can't find the container with id e99da8bf46db1cfe522607c169b1e83e969f15bc639f328518549e6b49008359
Sep 18 17:51:14.550550 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:14.550430    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z" event={"ID":"7bafd0b2-409e-46ba-a0b8-9391a248ab9f","Type":"ContainerStarted","Data":"e99da8bf46db1cfe522607c169b1e83e969f15bc639f328518549e6b49008359"}
Sep 18 17:51:18.522589 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:18.522557    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gtq6p"
Sep 18 17:51:25.089662 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:25.089626    2568 patch_prober.go:28] interesting pod/image-registry-9568558f4-kz48x container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 18 17:51:25.090133 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:25.089684    2568 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-9568558f4-kz48x" podUID="46939250-033d-42f7-a8b8-1330f875a69b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 17:51:25.361828 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:25.361801    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57d6b6fd-d5sdt"]
Sep 18 17:51:25.594595 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:25.594551    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z" event={"ID":"7bafd0b2-409e-46ba-a0b8-9391a248ab9f","Type":"ContainerStarted","Data":"456fe51266b920764ee8e5657505ef428e148bc0790a723c7a283ed3366253fe"}
Sep 18 17:51:25.594796 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:25.594624    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z" event={"ID":"7bafd0b2-409e-46ba-a0b8-9391a248ab9f","Type":"ContainerStarted","Data":"1d046fa8015582285c59e01db88f4c8f13d41dbbbefa57bd77d4e3c20ee23eae"}
Sep 18 17:51:25.596109 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:25.596079    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-sj7s5" event={"ID":"408c606b-e5cc-4afc-9a3d-afcab63ba455","Type":"ContainerStarted","Data":"59be45022ab4af39c13369c314658b8172753cccc4346783b1411515c44d5a89"}
Sep 18 17:51:25.596245 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:25.596228    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-sj7s5"
Sep 18 17:51:25.597832 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:25.597803    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d6b6fd-d5sdt" event={"ID":"bc594e74-714a-4959-95c6-cb8a6d9979cb","Type":"ContainerStarted","Data":"4b8685544049ae8dda44dbb2b3f9484568711c290f1c0315ca236a5eb005eaf5"}
Sep 18 17:51:25.598057 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:25.598036    2568 patch_prober.go:28] interesting pod/downloads-6594f4c75-sj7s5 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.13:8080/\": dial tcp 10.134.0.13:8080: connect: connection refused" start-of-body=
Sep 18 17:51:25.598104 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:25.598082    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-6594f4c75-sj7s5" podUID="408c606b-e5cc-4afc-9a3d-afcab63ba455" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.13:8080/\": dial tcp 10.134.0.13:8080: connect: connection refused"
Sep 18 17:51:25.618656 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:25.618435    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-hs89z" podStartSLOduration=2.654648649 podStartE2EDuration="13.618419153s" totalImagesPullingTime="10.963770504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:51:25.618301999 +0000 UTC m=+89.696105898" watchObservedRunningTime="2026-09-18 17:51:25.618419153 +0000 UTC m=+89.696223049"
Sep 18 17:51:25.654657 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:25.654602    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-sj7s5" podStartSLOduration=2.198264768 podStartE2EDuration="21.654582864s" totalImagesPullingTime="19.456318096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:51:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:51:25.653720841 +0000 UTC m=+89.731524740" watchObservedRunningTime="2026-09-18 17:51:25.654582864 +0000 UTC m=+89.732386764"
Sep 18 17:51:25.681113 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:25.681052    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57d6b6fd-d5sdt" podStartSLOduration=4.840682052 podStartE2EDuration="18.681030316s" totalImagesPullingTime="13.840348264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:51:25.679939708 +0000 UTC m=+89.757743608" watchObservedRunningTime="2026-09-18 17:51:25.681030316 +0000 UTC m=+89.758834218"
Sep 18 17:51:26.621463 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:26.621431    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-sj7s5"
Sep 18 17:51:27.494630 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.494598    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-9568558f4-kz48x"
Sep 18 17:51:27.600429 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.600395    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9qbhv"]
Sep 18 17:51:27.628122 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.628087    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.631972 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.631936    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gg4sf\""
Sep 18 17:51:27.632207 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.632166    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 17:51:27.632405 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.632346    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 17:51:27.634814 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.634712    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 17:51:27.756519 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.756432    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b29ebc8-1f18-4d33-be23-75eec038322b-root\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.756519 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.756486    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-wtmp\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.756763 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.756521    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-accelerators-collector-config\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.756763 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.756556    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-tls\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.756763 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.756581    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-textfile\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.756763 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.756613    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b29ebc8-1f18-4d33-be23-75eec038322b-sys\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.756763 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.756685    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b29ebc8-1f18-4d33-be23-75eec038322b-metrics-client-ca\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.756763 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.756736    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.757066 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.756804    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vrvp\" (UniqueName: \"kubernetes.io/projected/7b29ebc8-1f18-4d33-be23-75eec038322b-kube-api-access-8vrvp\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.857532 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.857493    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-wtmp\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.857718 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.857537    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-accelerators-collector-config\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.857718 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.857583    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-tls\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.857718 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.857611    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-textfile\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.857718 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.857649    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b29ebc8-1f18-4d33-be23-75eec038322b-sys\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.857718 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.857679    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b29ebc8-1f18-4d33-be23-75eec038322b-metrics-client-ca\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.857718 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.857693    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-wtmp\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.857718 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.857716    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.858584 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.857763    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vrvp\" (UniqueName: \"kubernetes.io/projected/7b29ebc8-1f18-4d33-be23-75eec038322b-kube-api-access-8vrvp\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.858584 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.857809    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b29ebc8-1f18-4d33-be23-75eec038322b-root\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.858584 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.857882    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b29ebc8-1f18-4d33-be23-75eec038322b-root\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.858584 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.857983    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b29ebc8-1f18-4d33-be23-75eec038322b-sys\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.858584 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:51:27.858003    2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 18 17:51:27.858584 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:51:27.858074    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-tls podName:7b29ebc8-1f18-4d33-be23-75eec038322b nodeName:}" failed. No retries permitted until 2026-09-18 17:51:28.358052586 +0000 UTC m=+92.435856463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-tls") pod "node-exporter-9qbhv" (UID: "7b29ebc8-1f18-4d33-be23-75eec038322b") : secret "node-exporter-tls" not found
Sep 18 17:51:27.858584 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.858227    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-textfile\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.858584 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.858336    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-accelerators-collector-config\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.858584 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.858541    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b29ebc8-1f18-4d33-be23-75eec038322b-metrics-client-ca\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.860606 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.860584    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:27.872396 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:27.872363    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vrvp\" (UniqueName: \"kubernetes.io/projected/7b29ebc8-1f18-4d33-be23-75eec038322b-kube-api-access-8vrvp\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:28.328007 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.327963    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:28.361433 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.361397    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-tls\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:28.364041 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.364007    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b29ebc8-1f18-4d33-be23-75eec038322b-node-exporter-tls\") pod \"node-exporter-9qbhv\" (UID: \"7b29ebc8-1f18-4d33-be23-75eec038322b\") " pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:28.541858 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.541817    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9qbhv"
Sep 18 17:51:28.577440 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:51:28.577378    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b29ebc8_1f18_4d33_be23_75eec038322b.slice/crio-6c807e2fdb075b5ee40d2ac643c4204b6ed8d0ccf99e10179b8c9ce72eed04f7 WatchSource:0}: Error finding container 6c807e2fdb075b5ee40d2ac643c4204b6ed8d0ccf99e10179b8c9ce72eed04f7: Status 404 returned error can't find the container with id 6c807e2fdb075b5ee40d2ac643c4204b6ed8d0ccf99e10179b8c9ce72eed04f7
Sep 18 17:51:28.612916 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.612870    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9qbhv" event={"ID":"7b29ebc8-1f18-4d33-be23-75eec038322b","Type":"ContainerStarted","Data":"6c807e2fdb075b5ee40d2ac643c4204b6ed8d0ccf99e10179b8c9ce72eed04f7"}
Sep 18 17:51:28.630056 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.630023    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 17:51:28.664351 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.664317    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.666259 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.666233    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 17:51:28.668266 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.668237    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 18 17:51:28.669695 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.669666    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-j7brw\""
Sep 18 17:51:28.669976 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.669957    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 18 17:51:28.670248 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.670228    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 18 17:51:28.670540 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.670520    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 18 17:51:28.671319 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.671297    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 18 17:51:28.671585 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.671560    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 18 17:51:28.672383 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.671788    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 18 17:51:28.672383 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.672134    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 18 17:51:28.672535 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.672434    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 18 17:51:28.764226 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.764189    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.764428 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.764238    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.764428 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.764281    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.764428 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.764313    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-config-volume\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.764428 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.764340    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.764428 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.764374    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xwxl\" (UniqueName: \"kubernetes.io/projected/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-kube-api-access-8xwxl\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.764428 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.764405    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.764746 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.764490    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.764746 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.764537    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-config-out\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.764746 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.764562    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-web-config\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.764746 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.764628    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-tls-assets\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.764746 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.764651    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.764746 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.764678    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.865695    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.865760    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-config-out\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.865796    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-web-config\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.865856    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-tls-assets\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.865907    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.865941    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.865999    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.866039    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.866072    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.866110    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-config-volume\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.866145    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.866208    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xwxl\" (UniqueName: \"kubernetes.io/projected/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-kube-api-access-8xwxl\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.866244    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:51:28.867736    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-alertmanager-trusted-ca-bundle podName:154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad nodeName:}" failed. No retries permitted until 2026-09-18 17:51:29.367707457 +0000 UTC m=+93.445511357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad") : configmap references non-existent config key: ca-bundle.crt
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.869031    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.872980 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:51:28.869204    2568 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 18 17:51:28.873946 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:51:28.869276    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-main-tls podName:154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad nodeName:}" failed. No retries permitted until 2026-09-18 17:51:29.369255482 +0000 UTC m=+93.447059376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad") : secret "alertmanager-main-tls" not found
Sep 18 17:51:28.873946 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.870673    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-web-config\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.873946 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.871327    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-config-out\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.873946 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.872461    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.875437 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.875324    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.877060 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.877011    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-config-volume\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.877400 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.877326    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.877400 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.877370    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-tls-assets\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.877846 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.877818    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.878529 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.878506    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:28.881480 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:28.881451    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xwxl\" (UniqueName: \"kubernetes.io/projected/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-kube-api-access-8xwxl\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:29.372313 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:29.372276    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:29.372511 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:29.372336    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:29.373164 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:29.373139    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:29.375080 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:29.375060    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:29.578084 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:29.578033    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:51:30.057275 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.057206    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 17:51:30.236570 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:51:30.236523    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod154ca6f5_1bc5_4a41_9f60_b23ca8ca15ad.slice/crio-f53c995e4c54820e450772a22b484e02f367c0ae650bf4de468da979f89203a0 WatchSource:0}: Error finding container f53c995e4c54820e450772a22b484e02f367c0ae650bf4de468da979f89203a0: Status 404 returned error can't find the container with id f53c995e4c54820e450772a22b484e02f367c0ae650bf4de468da979f89203a0
Sep 18 17:51:30.624303 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.624217    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad","Type":"ContainerStarted","Data":"f53c995e4c54820e450772a22b484e02f367c0ae650bf4de468da979f89203a0"}
Sep 18 17:51:30.626382 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.626348    2568 generic.go:356] "Generic (PLEG): container finished" podID="7b29ebc8-1f18-4d33-be23-75eec038322b" containerID="33239565bf974a3ee648e01d0b8422316855069660ddb52f79a394fbf7133e9a" exitCode=0
Sep 18 17:51:30.626502 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.626394    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9qbhv" event={"ID":"7b29ebc8-1f18-4d33-be23-75eec038322b","Type":"ContainerDied","Data":"33239565bf974a3ee648e01d0b8422316855069660ddb52f79a394fbf7133e9a"}
Sep 18 17:51:30.673328 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.673301    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"]
Sep 18 17:51:30.696661 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.696627    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.699746 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.699717    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 18 17:51:30.702164 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.702135    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 18 17:51:30.703502 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.703480    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-f8a1u7sfjkus\""
Sep 18 17:51:30.703613 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.703521    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 18 17:51:30.703613 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.703551    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 18 17:51:30.704160 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.704141    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 18 17:51:30.706010 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.705989    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lln46\""
Sep 18 17:51:30.745085 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.745055    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"]
Sep 18 17:51:30.787015 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.786887    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.787015 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.786956    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da6ebfa0-2312-4815-bc5a-a45adafdd82a-metrics-client-ca\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.787153 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.787139    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.787238 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.787169    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.787289 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.787260    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-grpc-tls\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.787351 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.787334    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-thanos-querier-tls\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.787408 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.787364    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.787460 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.787411    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbpvb\" (UniqueName: \"kubernetes.io/projected/da6ebfa0-2312-4815-bc5a-a45adafdd82a-kube-api-access-xbpvb\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.888820 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.888739    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-grpc-tls\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.888820 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.888795    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-thanos-querier-tls\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.889022 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.888823    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.889022 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.888862    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbpvb\" (UniqueName: \"kubernetes.io/projected/da6ebfa0-2312-4815-bc5a-a45adafdd82a-kube-api-access-xbpvb\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.889022 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.888897    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.889022 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.888933    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da6ebfa0-2312-4815-bc5a-a45adafdd82a-metrics-client-ca\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.889022 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.888984    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.889022 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.889009    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.891452 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.890041    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da6ebfa0-2312-4815-bc5a-a45adafdd82a-metrics-client-ca\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.892687 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.892627    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.893257 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.892808    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.893257 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.893220    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.893855 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.893806    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.895855 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.895031    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-grpc-tls\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.895855 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.895668    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/da6ebfa0-2312-4815-bc5a-a45adafdd82a-secret-thanos-querier-tls\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:30.915366 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:30.915214    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbpvb\" (UniqueName: \"kubernetes.io/projected/da6ebfa0-2312-4815-bc5a-a45adafdd82a-kube-api-access-xbpvb\") pod \"thanos-querier-7b6cb5d5df-mpktj\" (UID: \"da6ebfa0-2312-4815-bc5a-a45adafdd82a\") " pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:31.016965 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:31.016636    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:31.364198 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:31.364144    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"]
Sep 18 17:51:31.368703 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:51:31.368659    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda6ebfa0_2312_4815_bc5a_a45adafdd82a.slice/crio-ed160091c59c0b797fa65df3d85b1634f60d9b6ac2c5bcacbee1e25a3896e1db WatchSource:0}: Error finding container ed160091c59c0b797fa65df3d85b1634f60d9b6ac2c5bcacbee1e25a3896e1db: Status 404 returned error can't find the container with id ed160091c59c0b797fa65df3d85b1634f60d9b6ac2c5bcacbee1e25a3896e1db
Sep 18 17:51:31.633642 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:31.633514    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9qbhv" event={"ID":"7b29ebc8-1f18-4d33-be23-75eec038322b","Type":"ContainerStarted","Data":"0833c371a5356d6998cf68b4e8f7af5733f4583338897fb600270f63a901863f"}
Sep 18 17:51:31.633642 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:31.633581    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9qbhv" event={"ID":"7b29ebc8-1f18-4d33-be23-75eec038322b","Type":"ContainerStarted","Data":"61546bc0699bad675e5e1fb808ce134467fed6732ebf667fda1298773759a9a3"}
Sep 18 17:51:31.636984 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:31.636948    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj" event={"ID":"da6ebfa0-2312-4815-bc5a-a45adafdd82a","Type":"ContainerStarted","Data":"ed160091c59c0b797fa65df3d85b1634f60d9b6ac2c5bcacbee1e25a3896e1db"}
Sep 18 17:51:32.644498 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:32.644398    2568 generic.go:356] "Generic (PLEG): container finished" podID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerID="c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891" exitCode=0
Sep 18 17:51:32.645001 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:32.644524    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad","Type":"ContainerDied","Data":"c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891"}
Sep 18 17:51:32.678438 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:32.676863    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9qbhv" podStartSLOduration=3.912333674 podStartE2EDuration="5.676759037s" totalImagesPullingTime="1.679197357s" totalInitContainerRuntime="85.228006ms" isStatefulPod=true podCreationTimestamp="2026-09-18 17:51:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:51:31.661048737 +0000 UTC m=+95.738852637" watchObservedRunningTime="2026-09-18 17:51:32.676759037 +0000 UTC m=+96.754562938"
Sep 18 17:51:33.493256 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.492227    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cd7b65677-k2mlc"]
Sep 18 17:51:33.517188 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.517143    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cd7b65677-k2mlc"]
Sep 18 17:51:33.517365 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.517292    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.526075 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.525948    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 18 17:51:33.614881 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.614832    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-oauth-serving-cert\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.614881 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.614886    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnhp6\" (UniqueName: \"kubernetes.io/projected/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-kube-api-access-rnhp6\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.615140 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.614922    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-serving-cert\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.615140 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.614963    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-trusted-ca-bundle\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.615140 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.615004    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-config\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.615140 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.615034    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-oauth-config\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.615140 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.615091    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-service-ca\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.716489 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.716442    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-trusted-ca-bundle\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.717085 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.716517    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-config\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.717085 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.716549    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-oauth-config\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.717085 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.716606    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-service-ca\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.717085 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.716660    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-oauth-serving-cert\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.717085 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.716683    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnhp6\" (UniqueName: \"kubernetes.io/projected/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-kube-api-access-rnhp6\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.717085 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.716719    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-serving-cert\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.718015 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.717522    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-trusted-ca-bundle\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.719322 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.719293    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-service-ca\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.719740 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.719718    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-config\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.720117 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.720092    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-oauth-config\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.720566 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.720548    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-serving-cert\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.720826 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.720806    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-oauth-serving-cert\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.728340 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.728272    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnhp6\" (UniqueName: \"kubernetes.io/projected/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-kube-api-access-rnhp6\") pod \"console-6cd7b65677-k2mlc\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") " pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:33.830947 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:33.830858    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:34.153075 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:34.153050    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cd7b65677-k2mlc"]
Sep 18 17:51:34.232956 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:51:34.232917    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d51b6d5_780b_40b5_9c92_6eb83cafceb3.slice/crio-b4b4a15a7d7701261f8caec69b5b40552ab693993274d7f82d3186d237f3f868 WatchSource:0}: Error finding container b4b4a15a7d7701261f8caec69b5b40552ab693993274d7f82d3186d237f3f868: Status 404 returned error can't find the container with id b4b4a15a7d7701261f8caec69b5b40552ab693993274d7f82d3186d237f3f868
Sep 18 17:51:34.660216 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:34.660152    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj" event={"ID":"da6ebfa0-2312-4815-bc5a-a45adafdd82a","Type":"ContainerStarted","Data":"d24205a9d9eb22a4774cb111d3bef887d48fe794d323a2fa3a49f8cd04dfd9ca"}
Sep 18 17:51:34.660406 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:34.660280    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj" event={"ID":"da6ebfa0-2312-4815-bc5a-a45adafdd82a","Type":"ContainerStarted","Data":"5c2bbbbf3e718d4220d4a4c925ffc3925f4ed89735420067d982f0d0de7d154b"}
Sep 18 17:51:34.662654 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:34.662618    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd7b65677-k2mlc" event={"ID":"0d51b6d5-780b-40b5-9c92-6eb83cafceb3","Type":"ContainerStarted","Data":"8a0198e2817fe875b751026901a38e257f4f9d2ea08ce079f791a7f352cde002"}
Sep 18 17:51:34.662790 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:34.662669    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd7b65677-k2mlc" event={"ID":"0d51b6d5-780b-40b5-9c92-6eb83cafceb3","Type":"ContainerStarted","Data":"b4b4a15a7d7701261f8caec69b5b40552ab693993274d7f82d3186d237f3f868"}
Sep 18 17:51:34.724139 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:34.724085    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cd7b65677-k2mlc" podStartSLOduration=1.724064304 podStartE2EDuration="1.724064304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:51:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:51:34.721928364 +0000 UTC m=+98.799732273" watchObservedRunningTime="2026-09-18 17:51:34.724064304 +0000 UTC m=+98.801868202"
Sep 18 17:51:35.669066 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:35.669029    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj" event={"ID":"da6ebfa0-2312-4815-bc5a-a45adafdd82a","Type":"ContainerStarted","Data":"25388dfa4072d395920788c3db2eaf6601c54a875b15c63a2e11150d962d0d85"}
Sep 18 17:51:36.676781 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:36.676755    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad","Type":"ContainerStarted","Data":"8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b"}
Sep 18 17:51:36.677086 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:36.676818    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad","Type":"ContainerStarted","Data":"887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef"}
Sep 18 17:51:36.677086 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:36.676843    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad","Type":"ContainerStarted","Data":"59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c"}
Sep 18 17:51:36.677086 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:36.676866    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad","Type":"ContainerStarted","Data":"c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d"}
Sep 18 17:51:37.683580 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:37.683541    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj" event={"ID":"da6ebfa0-2312-4815-bc5a-a45adafdd82a","Type":"ContainerStarted","Data":"6e54d43d848e5a3cb414aa0d9018905549c0f9a12cdf86dd0ce2cb2788376d9f"}
Sep 18 17:51:37.684037 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:37.683614    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj" event={"ID":"da6ebfa0-2312-4815-bc5a-a45adafdd82a","Type":"ContainerStarted","Data":"f8ac0dc6c30fe33968149952b0322225a1cc316527cbcbdb2803acc0f2dddcb7"}
Sep 18 17:51:37.684037 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:37.683664    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj" event={"ID":"da6ebfa0-2312-4815-bc5a-a45adafdd82a","Type":"ContainerStarted","Data":"41cf45ec0b4699384603a50af3b8a2f9531875e7fec48aeaf5850f522ef6ea4c"}
Sep 18 17:51:37.684037 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:37.683703    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:37.686995 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:37.686970    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad","Type":"ContainerStarted","Data":"d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af"}
Sep 18 17:51:37.687083 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:37.687024    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad","Type":"ContainerStarted","Data":"8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2"}
Sep 18 17:51:37.721946 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:37.721901    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj" podStartSLOduration=2.900384702 podStartE2EDuration="7.7218851s" totalImagesPullingTime="4.821500398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:51:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:51:37.719941305 +0000 UTC m=+101.797745205" watchObservedRunningTime="2026-09-18 17:51:37.7218851 +0000 UTC m=+101.799689035"
Sep 18 17:51:37.762726 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:37.762671    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.659553937 podStartE2EDuration="9.762654654s" totalImagesPullingTime="5.080487418s" totalInitContainerRuntime="22.613299ms" isStatefulPod=false podCreationTimestamp="2026-09-18 17:51:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:51:37.762224412 +0000 UTC m=+101.840028314" watchObservedRunningTime="2026-09-18 17:51:37.762654654 +0000 UTC m=+101.840458552"
Sep 18 17:51:42.522737 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:42.522703    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dhmwn"
Sep 18 17:51:43.698206 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:43.698149    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7b6cb5d5df-mpktj"
Sep 18 17:51:43.831232 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:43.831196    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:43.831392 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:43.831260    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:43.836074 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:43.836051    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:44.719208 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:44.719163    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:51:51.624140 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.624091    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-57d6b6fd-d5sdt" podUID="bc594e74-714a-4959-95c6-cb8a6d9979cb" containerName="console" containerID="cri-o://4b8685544049ae8dda44dbb2b3f9484568711c290f1c0315ca236a5eb005eaf5" gracePeriod=15
Sep 18 17:51:51.745355 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.745324    2568 generic.go:356] "Generic (PLEG): container finished" podID="bc594e74-714a-4959-95c6-cb8a6d9979cb" containerID="4b8685544049ae8dda44dbb2b3f9484568711c290f1c0315ca236a5eb005eaf5" exitCode=0
Sep 18 17:51:51.745517 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.745399    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d6b6fd-d5sdt" event={"ID":"bc594e74-714a-4959-95c6-cb8a6d9979cb","Type":"ContainerDied","Data":"4b8685544049ae8dda44dbb2b3f9484568711c290f1c0315ca236a5eb005eaf5"}
Sep 18 17:51:51.894733 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.894670    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:51.968335 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.968298    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qvpm\" (UniqueName: \"kubernetes.io/projected/bc594e74-714a-4959-95c6-cb8a6d9979cb-kube-api-access-2qvpm\") pod \"bc594e74-714a-4959-95c6-cb8a6d9979cb\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") "
Sep 18 17:51:51.968509 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.968347    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-serving-cert\") pod \"bc594e74-714a-4959-95c6-cb8a6d9979cb\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") "
Sep 18 17:51:51.968509 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.968367    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-oauth-config\") pod \"bc594e74-714a-4959-95c6-cb8a6d9979cb\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") "
Sep 18 17:51:51.968509 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.968389    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-oauth-serving-cert\") pod \"bc594e74-714a-4959-95c6-cb8a6d9979cb\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") "
Sep 18 17:51:51.968663 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.968582    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-config\") pod \"bc594e74-714a-4959-95c6-cb8a6d9979cb\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") "
Sep 18 17:51:51.968718 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.968670    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-service-ca\") pod \"bc594e74-714a-4959-95c6-cb8a6d9979cb\" (UID: \"bc594e74-714a-4959-95c6-cb8a6d9979cb\") "
Sep 18 17:51:51.968775 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.968750    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bc594e74-714a-4959-95c6-cb8a6d9979cb" (UID: "bc594e74-714a-4959-95c6-cb8a6d9979cb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:51:51.968995 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.968970    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-config" (OuterVolumeSpecName: "console-config") pod "bc594e74-714a-4959-95c6-cb8a6d9979cb" (UID: "bc594e74-714a-4959-95c6-cb8a6d9979cb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:51:51.969107 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.969037    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-service-ca" (OuterVolumeSpecName: "service-ca") pod "bc594e74-714a-4959-95c6-cb8a6d9979cb" (UID: "bc594e74-714a-4959-95c6-cb8a6d9979cb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:51:51.969107 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.969048    2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-oauth-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:51:51.969107 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.969066    2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:51:51.970883 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.970854    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bc594e74-714a-4959-95c6-cb8a6d9979cb" (UID: "bc594e74-714a-4959-95c6-cb8a6d9979cb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:51:51.971040 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.971020    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bc594e74-714a-4959-95c6-cb8a6d9979cb" (UID: "bc594e74-714a-4959-95c6-cb8a6d9979cb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:51:51.971157 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:51.971143    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc594e74-714a-4959-95c6-cb8a6d9979cb-kube-api-access-2qvpm" (OuterVolumeSpecName: "kube-api-access-2qvpm") pod "bc594e74-714a-4959-95c6-cb8a6d9979cb" (UID: "bc594e74-714a-4959-95c6-cb8a6d9979cb"). InnerVolumeSpecName "kube-api-access-2qvpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 17:51:52.070147 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:52.070115    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qvpm\" (UniqueName: \"kubernetes.io/projected/bc594e74-714a-4959-95c6-cb8a6d9979cb-kube-api-access-2qvpm\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:51:52.070147 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:52.070140    2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:51:52.070147 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:52.070149    2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc594e74-714a-4959-95c6-cb8a6d9979cb-console-oauth-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:51:52.070379 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:52.070158    2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc594e74-714a-4959-95c6-cb8a6d9979cb-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:51:52.750078 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:52.750052    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d6b6fd-d5sdt"
Sep 18 17:51:52.750078 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:52.750056    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d6b6fd-d5sdt" event={"ID":"bc594e74-714a-4959-95c6-cb8a6d9979cb","Type":"ContainerDied","Data":"70edb6a614a606809356862b53f595781682710924528695069dbe9389e4cabe"}
Sep 18 17:51:52.750582 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:52.750109    2568 scope.go:117] "RemoveContainer" containerID="4b8685544049ae8dda44dbb2b3f9484568711c290f1c0315ca236a5eb005eaf5"
Sep 18 17:51:52.767268 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:52.767241    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57d6b6fd-d5sdt"]
Sep 18 17:51:52.771973 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:52.771951    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57d6b6fd-d5sdt"]
Sep 18 17:51:54.043540 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:51:54.043504    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc594e74-714a-4959-95c6-cb8a6d9979cb" path="/var/lib/kubelet/pods/bc594e74-714a-4959-95c6-cb8a6d9979cb/volumes"
Sep 18 17:52:48.351278 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:48.351170    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 17:52:48.351819 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:48.351555    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="alertmanager" containerID="cri-o://c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d" gracePeriod=120
Sep 18 17:52:48.351819 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:48.351574    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="prom-label-proxy" containerID="cri-o://d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af" gracePeriod=120
Sep 18 17:52:48.351819 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:48.351590    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="kube-rbac-proxy-web" containerID="cri-o://887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef" gracePeriod=120
Sep 18 17:52:48.351819 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:48.351620    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="config-reloader" containerID="cri-o://59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c" gracePeriod=120
Sep 18 17:52:48.351819 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:48.351545    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="kube-rbac-proxy-metric" containerID="cri-o://8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2" gracePeriod=120
Sep 18 17:52:48.351819 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:48.351704    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="kube-rbac-proxy" containerID="cri-o://8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b" gracePeriod=120
Sep 18 17:52:48.380698 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:52:48.380669    2568 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod154ca6f5_1bc5_4a41_9f60_b23ca8ca15ad.slice/crio-8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod154ca6f5_1bc5_4a41_9f60_b23ca8ca15ad.slice/crio-d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af.scope\": RecentStats: unable to find data in memory cache]"
Sep 18 17:52:48.969348 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:48.969314    2568 generic.go:356] "Generic (PLEG): container finished" podID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerID="d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af" exitCode=0
Sep 18 17:52:48.969348 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:48.969339    2568 generic.go:356] "Generic (PLEG): container finished" podID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerID="8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b" exitCode=0
Sep 18 17:52:48.969348 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:48.969345    2568 generic.go:356] "Generic (PLEG): container finished" podID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerID="59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c" exitCode=0
Sep 18 17:52:48.969348 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:48.969352    2568 generic.go:356] "Generic (PLEG): container finished" podID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerID="c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d" exitCode=0
Sep 18 17:52:48.969608 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:48.969386    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad","Type":"ContainerDied","Data":"d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af"}
Sep 18 17:52:48.969608 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:48.969450    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad","Type":"ContainerDied","Data":"8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b"}
Sep 18 17:52:48.969608 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:48.969477    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad","Type":"ContainerDied","Data":"59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c"}
Sep 18 17:52:48.969608 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:48.969502    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad","Type":"ContainerDied","Data":"c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d"}
Sep 18 17:52:49.596918 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.596896    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:49.621069 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.621047    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy-web\") pod \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") "
Sep 18 17:52:49.621211 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.621078    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy\") pod \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") "
Sep 18 17:52:49.621211 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.621103    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-cluster-tls-config\") pod \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") "
Sep 18 17:52:49.621211 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.621138    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-web-config\") pod \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") "
Sep 18 17:52:49.621211 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.621168    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-config-volume\") pod \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") "
Sep 18 17:52:49.621211 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.621208    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-config-out\") pod \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") "
Sep 18 17:52:49.621474 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.621232    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-main-tls\") pod \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") "
Sep 18 17:52:49.621474 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.621280    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-metrics-client-ca\") pod \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") "
Sep 18 17:52:49.621474 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.621329    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xwxl\" (UniqueName: \"kubernetes.io/projected/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-kube-api-access-8xwxl\") pod \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") "
Sep 18 17:52:49.621474 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.621353    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-tls-assets\") pod \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") "
Sep 18 17:52:49.621474 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.621382    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy-metric\") pod \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") "
Sep 18 17:52:49.621474 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.621412    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-alertmanager-trusted-ca-bundle\") pod \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") "
Sep 18 17:52:49.621474 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.621447    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-alertmanager-main-db\") pod \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\" (UID: \"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad\") "
Sep 18 17:52:49.622007 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.621931    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" (UID: "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 17:52:49.622695 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.622663    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" (UID: "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:52:49.622886 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.622862    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" (UID: "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:52:49.624436 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.624408    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" (UID: "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:52:49.624532 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.624502    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" (UID: "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:52:49.624700 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.624661    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-config-out" (OuterVolumeSpecName: "config-out") pod "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" (UID: "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 17:52:49.624700 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.624677    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-config-volume" (OuterVolumeSpecName: "config-volume") pod "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" (UID: "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:52:49.625517 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.625495    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" (UID: "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:52:49.625517 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.625501    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-kube-api-access-8xwxl" (OuterVolumeSpecName: "kube-api-access-8xwxl") pod "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" (UID: "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad"). InnerVolumeSpecName "kube-api-access-8xwxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 17:52:49.625627 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.625541    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" (UID: "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:52:49.626129 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.626112    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" (UID: "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 17:52:49.629145 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.629103    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" (UID: "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:52:49.635104 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.635084    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-web-config" (OuterVolumeSpecName: "web-config") pod "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" (UID: "154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:52:49.722222 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.722119    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:52:49.722222 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.722149    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:52:49.722222 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.722163    2568 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-cluster-tls-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:52:49.722222 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.722190    2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-web-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:52:49.722222 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.722202    2568 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:52:49.722222 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.722214    2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-config-out\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:52:49.722222 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.722226    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-main-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:52:49.722545 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.722237    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-metrics-client-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:52:49.722545 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.722249    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xwxl\" (UniqueName: \"kubernetes.io/projected/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-kube-api-access-8xwxl\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:52:49.722545 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.722259    2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-tls-assets\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:52:49.722545 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.722271    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:52:49.722545 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.722283    2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:52:49.722545 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.722296    2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad-alertmanager-main-db\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:52:49.975783 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.975685    2568 generic.go:356] "Generic (PLEG): container finished" podID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerID="8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2" exitCode=0
Sep 18 17:52:49.975783 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.975711    2568 generic.go:356] "Generic (PLEG): container finished" podID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerID="887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef" exitCode=0
Sep 18 17:52:49.975995 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.975776    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad","Type":"ContainerDied","Data":"8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2"}
Sep 18 17:52:49.975995 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.975791    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:49.975995 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.975874    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad","Type":"ContainerDied","Data":"887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef"}
Sep 18 17:52:49.975995 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.975902    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad","Type":"ContainerDied","Data":"f53c995e4c54820e450772a22b484e02f367c0ae650bf4de468da979f89203a0"}
Sep 18 17:52:49.975995 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.975935    2568 scope.go:117] "RemoveContainer" containerID="d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af"
Sep 18 17:52:49.983524 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.983411    2568 scope.go:117] "RemoveContainer" containerID="8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2"
Sep 18 17:52:49.989952 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.989934    2568 scope.go:117] "RemoveContainer" containerID="8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b"
Sep 18 17:52:49.996268 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:49.996214    2568 scope.go:117] "RemoveContainer" containerID="887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef"
Sep 18 17:52:50.002279 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.002257    2568 scope.go:117] "RemoveContainer" containerID="59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c"
Sep 18 17:52:50.004429 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.004409    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 17:52:50.009168 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.009147    2568 scope.go:117] "RemoveContainer" containerID="c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d"
Sep 18 17:52:50.011721 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.011699    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 17:52:50.015920 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.015903    2568 scope.go:117] "RemoveContainer" containerID="c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891"
Sep 18 17:52:50.022163 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.022149    2568 scope.go:117] "RemoveContainer" containerID="d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af"
Sep 18 17:52:50.022464 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:52:50.022439    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af\": container with ID starting with d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af not found: ID does not exist" containerID="d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af"
Sep 18 17:52:50.022514 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.022466    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af"} err="failed to get container status \"d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af\": rpc error: code = NotFound desc = could not find container \"d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af\": container with ID starting with d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af not found: ID does not exist"
Sep 18 17:52:50.022514 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.022496    2568 scope.go:117] "RemoveContainer" containerID="8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2"
Sep 18 17:52:50.022715 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:52:50.022698    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2\": container with ID starting with 8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2 not found: ID does not exist" containerID="8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2"
Sep 18 17:52:50.022754 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.022721    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2"} err="failed to get container status \"8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2\": rpc error: code = NotFound desc = could not find container \"8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2\": container with ID starting with 8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2 not found: ID does not exist"
Sep 18 17:52:50.022754 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.022742    2568 scope.go:117] "RemoveContainer" containerID="8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b"
Sep 18 17:52:50.022948 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:52:50.022932    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b\": container with ID starting with 8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b not found: ID does not exist" containerID="8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b"
Sep 18 17:52:50.022988 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.022952    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b"} err="failed to get container status \"8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b\": rpc error: code = NotFound desc = could not find container \"8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b\": container with ID starting with 8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b not found: ID does not exist"
Sep 18 17:52:50.022988 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.022964    2568 scope.go:117] "RemoveContainer" containerID="887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef"
Sep 18 17:52:50.023154 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:52:50.023139    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef\": container with ID starting with 887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef not found: ID does not exist" containerID="887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef"
Sep 18 17:52:50.023213 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.023156    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef"} err="failed to get container status \"887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef\": rpc error: code = NotFound desc = could not find container \"887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef\": container with ID starting with 887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef not found: ID does not exist"
Sep 18 17:52:50.023213 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.023167    2568 scope.go:117] "RemoveContainer" containerID="59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c"
Sep 18 17:52:50.023388 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:52:50.023372    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c\": container with ID starting with 59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c not found: ID does not exist" containerID="59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c"
Sep 18 17:52:50.023429 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.023391    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c"} err="failed to get container status \"59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c\": rpc error: code = NotFound desc = could not find container \"59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c\": container with ID starting with 59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c not found: ID does not exist"
Sep 18 17:52:50.023429 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.023402    2568 scope.go:117] "RemoveContainer" containerID="c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d"
Sep 18 17:52:50.023585 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:52:50.023567    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d\": container with ID starting with c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d not found: ID does not exist" containerID="c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d"
Sep 18 17:52:50.023622 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.023589    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d"} err="failed to get container status \"c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d\": rpc error: code = NotFound desc = could not find container \"c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d\": container with ID starting with c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d not found: ID does not exist"
Sep 18 17:52:50.023622 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.023603    2568 scope.go:117] "RemoveContainer" containerID="c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891"
Sep 18 17:52:50.023779 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:52:50.023764    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891\": container with ID starting with c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891 not found: ID does not exist" containerID="c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891"
Sep 18 17:52:50.023816 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.023782    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891"} err="failed to get container status \"c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891\": rpc error: code = NotFound desc = could not find container \"c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891\": container with ID starting with c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891 not found: ID does not exist"
Sep 18 17:52:50.023816 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.023793    2568 scope.go:117] "RemoveContainer" containerID="d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af"
Sep 18 17:52:50.024004 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.023983    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af"} err="failed to get container status \"d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af\": rpc error: code = NotFound desc = could not find container \"d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af\": container with ID starting with d8c7abfac355ea6568c3835dd284590317e0f1677b5f1a2b8ac7540e815fa0af not found: ID does not exist"
Sep 18 17:52:50.024046 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.024006    2568 scope.go:117] "RemoveContainer" containerID="8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2"
Sep 18 17:52:50.024250 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.024234    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2"} err="failed to get container status \"8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2\": rpc error: code = NotFound desc = could not find container \"8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2\": container with ID starting with 8fd0788c4d2a5836cda17f0cd6af9f9b06ed68ffd8fc2b253ae5daff5bb3d5b2 not found: ID does not exist"
Sep 18 17:52:50.024250 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.024251    2568 scope.go:117] "RemoveContainer" containerID="8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b"
Sep 18 17:52:50.024480 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.024462    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b"} err="failed to get container status \"8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b\": rpc error: code = NotFound desc = could not find container \"8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b\": container with ID starting with 8c088eb450fd77f1dc6703ca3b31210ca8e8a55b008615264648b343d770550b not found: ID does not exist"
Sep 18 17:52:50.024522 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.024480    2568 scope.go:117] "RemoveContainer" containerID="887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef"
Sep 18 17:52:50.024688 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.024672    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef"} err="failed to get container status \"887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef\": rpc error: code = NotFound desc = could not find container \"887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef\": container with ID starting with 887d8d015b888b8f095c39da1f3067adbf43aaa7c6ab5cfe93fcd4b635fc12ef not found: ID does not exist"
Sep 18 17:52:50.024688 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.024687    2568 scope.go:117] "RemoveContainer" containerID="59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c"
Sep 18 17:52:50.024881 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.024867    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c"} err="failed to get container status \"59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c\": rpc error: code = NotFound desc = could not find container \"59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c\": container with ID starting with 59ce3a8d0c2ec8c3b0bf3d938bfe5e824b0462fc56d2d22f128bf8aa771e512c not found: ID does not exist"
Sep 18 17:52:50.024921 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.024882    2568 scope.go:117] "RemoveContainer" containerID="c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d"
Sep 18 17:52:50.025092 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.025073    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d"} err="failed to get container status \"c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d\": rpc error: code = NotFound desc = could not find container \"c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d\": container with ID starting with c6f92edfdda69b06be0113688498d9a8ae0ad710b59ba8824002bee54375958d not found: ID does not exist"
Sep 18 17:52:50.025143 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.025092    2568 scope.go:117] "RemoveContainer" containerID="c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891"
Sep 18 17:52:50.025300 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.025284    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891"} err="failed to get container status \"c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891\": rpc error: code = NotFound desc = could not find container \"c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891\": container with ID starting with c50447513ff9f9b10cf51a1e075c52ed94ddfaf61ef7738991f98de7fdcd8891 not found: ID does not exist"
Sep 18 17:52:50.042909 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.042890    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" path="/var/lib/kubelet/pods/154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad/volumes"
Sep 18 17:52:50.050380 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050364    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 17:52:50.050629 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050618    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="prom-label-proxy"
Sep 18 17:52:50.050665 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050631    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="prom-label-proxy"
Sep 18 17:52:50.050665 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050639    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="config-reloader"
Sep 18 17:52:50.050665 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050645    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="config-reloader"
Sep 18 17:52:50.050665 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050654    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="kube-rbac-proxy-web"
Sep 18 17:52:50.050665 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050659    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="kube-rbac-proxy-web"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050667    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="kube-rbac-proxy"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050672    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="kube-rbac-proxy"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050677    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="alertmanager"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050682    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="alertmanager"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050691    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc594e74-714a-4959-95c6-cb8a6d9979cb" containerName="console"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050695    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc594e74-714a-4959-95c6-cb8a6d9979cb" containerName="console"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050700    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="kube-rbac-proxy-metric"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050705    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="kube-rbac-proxy-metric"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050712    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="init-config-reloader"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050717    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="init-config-reloader"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050761    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="alertmanager"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050767    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="prom-label-proxy"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050773    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc594e74-714a-4959-95c6-cb8a6d9979cb" containerName="console"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050778    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="kube-rbac-proxy-web"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050784    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="kube-rbac-proxy"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050790    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="config-reloader"
Sep 18 17:52:50.050805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.050795    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="154ca6f5-1bc5-4a41-9f60-b23ca8ca15ad" containerName="kube-rbac-proxy-metric"
Sep 18 17:52:50.054048 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.054034    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.058565 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.058547    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 18 17:52:50.058962 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.058944    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 18 17:52:50.059049 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.058951    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 18 17:52:50.059049 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.058996    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 18 17:52:50.059049 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.059023    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 18 17:52:50.059235 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.059023    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 18 17:52:50.059324 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.059309    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-j7brw\""
Sep 18 17:52:50.061640 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.061618    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 18 17:52:50.065722 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.065704    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 18 17:52:50.065844 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.065827    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 18 17:52:50.076318 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.076294    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 17:52:50.125543 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.125513    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.125664 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.125560    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.125753 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.125726    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.125804 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.125770    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1eb85964-aedf-40b4-bc35-c0ac16d68c10-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.125866 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.125799    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd88k\" (UniqueName: \"kubernetes.io/projected/1eb85964-aedf-40b4-bc35-c0ac16d68c10-kube-api-access-pd88k\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.125866 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.125860    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-config-volume\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.125968 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.125886    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.125968 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.125921    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1eb85964-aedf-40b4-bc35-c0ac16d68c10-config-out\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.125968 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.125948    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1eb85964-aedf-40b4-bc35-c0ac16d68c10-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.126123 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.125993    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eb85964-aedf-40b4-bc35-c0ac16d68c10-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.126123 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.126116    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1eb85964-aedf-40b4-bc35-c0ac16d68c10-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.126244 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.126133    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.126244 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.126158    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-web-config\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.227003 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.226931    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1eb85964-aedf-40b4-bc35-c0ac16d68c10-config-out\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.227003 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.226960    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1eb85964-aedf-40b4-bc35-c0ac16d68c10-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.227003 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.226980    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eb85964-aedf-40b4-bc35-c0ac16d68c10-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.227003 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.227002    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1eb85964-aedf-40b4-bc35-c0ac16d68c10-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.227306 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.227017    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.227306 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.227036    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-web-config\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.227306 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.227053    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.227306 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.227079    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.227306 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.227134    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.227306 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.227157    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1eb85964-aedf-40b4-bc35-c0ac16d68c10-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.227306 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.227201    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd88k\" (UniqueName: \"kubernetes.io/projected/1eb85964-aedf-40b4-bc35-c0ac16d68c10-kube-api-access-pd88k\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.227306 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.227231    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-config-volume\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.227306 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.227254    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.227831 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.227809    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1eb85964-aedf-40b4-bc35-c0ac16d68c10-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.228157 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.228005    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eb85964-aedf-40b4-bc35-c0ac16d68c10-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.228157 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.228062    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1eb85964-aedf-40b4-bc35-c0ac16d68c10-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.229901 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.229870    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1eb85964-aedf-40b4-bc35-c0ac16d68c10-config-out\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.230065 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.230042    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.230135 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.230105    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1eb85964-aedf-40b4-bc35-c0ac16d68c10-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.230135 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.230124    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.230428 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.230400    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-web-config\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.230566 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.230546    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.230709 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.230689    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.230836 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.230820    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.231901 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.231885    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1eb85964-aedf-40b4-bc35-c0ac16d68c10-config-volume\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.239824 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.239805    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd88k\" (UniqueName: \"kubernetes.io/projected/1eb85964-aedf-40b4-bc35-c0ac16d68c10-kube-api-access-pd88k\") pod \"alertmanager-main-0\" (UID: \"1eb85964-aedf-40b4-bc35-c0ac16d68c10\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.363534 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.363504    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 17:52:50.643776 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.643725    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 17:52:50.646345 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:52:50.646297    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eb85964_aedf_40b4_bc35_c0ac16d68c10.slice/crio-0aa6e3bf6f7cad680ea71763199a80c7187065f02b76d4140efaf531cdeb96d9 WatchSource:0}: Error finding container 0aa6e3bf6f7cad680ea71763199a80c7187065f02b76d4140efaf531cdeb96d9: Status 404 returned error can't find the container with id 0aa6e3bf6f7cad680ea71763199a80c7187065f02b76d4140efaf531cdeb96d9
Sep 18 17:52:50.983543 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.983512    2568 generic.go:356] "Generic (PLEG): container finished" podID="1eb85964-aedf-40b4-bc35-c0ac16d68c10" containerID="b11ee9f60db7b748efeef69c41c551b8d5f2685791cf8b654eb373e54b90739d" exitCode=0
Sep 18 17:52:50.983758 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.983611    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1eb85964-aedf-40b4-bc35-c0ac16d68c10","Type":"ContainerDied","Data":"b11ee9f60db7b748efeef69c41c551b8d5f2685791cf8b654eb373e54b90739d"}
Sep 18 17:52:50.983758 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:50.983689    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1eb85964-aedf-40b4-bc35-c0ac16d68c10","Type":"ContainerStarted","Data":"0aa6e3bf6f7cad680ea71763199a80c7187065f02b76d4140efaf531cdeb96d9"}
Sep 18 17:52:51.991279 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:51.991244    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1eb85964-aedf-40b4-bc35-c0ac16d68c10","Type":"ContainerStarted","Data":"e65aba37e7ee7cd7e9432c6da2c5fbbc177341e3552013ec796bdaef61393537"}
Sep 18 17:52:51.991687 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:51.991355    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1eb85964-aedf-40b4-bc35-c0ac16d68c10","Type":"ContainerStarted","Data":"e68c661fd258878ba92ed3508ca2084552bf862a0e58bef3e5e864bab10c4f66"}
Sep 18 17:52:51.991687 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:51.991401    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1eb85964-aedf-40b4-bc35-c0ac16d68c10","Type":"ContainerStarted","Data":"edba59e690993c6d928713c592e634816ba0953141037738bd095a564269343b"}
Sep 18 17:52:51.991687 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:51.991442    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1eb85964-aedf-40b4-bc35-c0ac16d68c10","Type":"ContainerStarted","Data":"d0653701b2ce56a9c770d142226733fb9936ac4a1f1cc3aed95706739d7c6194"}
Sep 18 17:52:51.991687 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:51.991478    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1eb85964-aedf-40b4-bc35-c0ac16d68c10","Type":"ContainerStarted","Data":"bf77d22f682bedf3569ea6a14b2ba1b67adcbf141e31a2116e0c92817b41d49c"}
Sep 18 17:52:51.991687 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:51.991515    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1eb85964-aedf-40b4-bc35-c0ac16d68c10","Type":"ContainerStarted","Data":"559efb552f9d2a49a876482e742f6e4626701627c3aa6c36d2f0c89acb234389"}
Sep 18 17:52:52.031721 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.031674    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.012274274 podStartE2EDuration="2.031662108s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.387834ms" isStatefulPod=false podCreationTimestamp="2026-09-18 17:52:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:52:52.027060152 +0000 UTC m=+176.104864074" watchObservedRunningTime="2026-09-18 17:52:52.031662108 +0000 UTC m=+176.109466007"
Sep 18 17:52:52.445533 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.445498    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-8646db9fbf-x72lb"]
Sep 18 17:52:52.447766 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.447738    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.450267 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.450242    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 18 17:52:52.450466 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.450454    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-z8bff\""
Sep 18 17:52:52.450654 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.450637    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 18 17:52:52.450837 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.450823    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 18 17:52:52.451189 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.451157    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 18 17:52:52.451277 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.451253    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 18 17:52:52.457061 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.457037    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 18 17:52:52.466813 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.466791    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8646db9fbf-x72lb"]
Sep 18 17:52:52.545026 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.544992    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-serving-certs-ca-bundle\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.545297 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.545277    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-secret-telemeter-client\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.545418 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.545404    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-telemeter-client-tls\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.545539 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.545523    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-metrics-client-ca\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.545654 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.545640    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-federate-client-tls\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.545762 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.545748    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.545899 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.545885    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.546017 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.546005    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9z88\" (UniqueName: \"kubernetes.io/projected/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-kube-api-access-l9z88\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.648882 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.647481    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.648882 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.647564    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9z88\" (UniqueName: \"kubernetes.io/projected/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-kube-api-access-l9z88\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.648882 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.647631    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-serving-certs-ca-bundle\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.648882 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.647670    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-secret-telemeter-client\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.648882 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.647696    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-telemeter-client-tls\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.648882 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.647728    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-metrics-client-ca\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.648882 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.647754    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-federate-client-tls\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.648882 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.647775    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.648882 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.648796    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.649580 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.649556    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-serving-certs-ca-bundle\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.649788 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.649765    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-metrics-client-ca\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.651553 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.651524    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-secret-telemeter-client\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.651777 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.651761    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.651843 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.651822    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-telemeter-client-tls\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.651877 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.651846    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-federate-client-tls\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.658817 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.658790    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9z88\" (UniqueName: \"kubernetes.io/projected/719fcb78-5c8d-4a1f-811b-2d899e76c7b1-kube-api-access-l9z88\") pod \"telemeter-client-8646db9fbf-x72lb\" (UID: \"719fcb78-5c8d-4a1f-811b-2d899e76c7b1\") " pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:52.758809 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:52.758722    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb"
Sep 18 17:52:53.056503 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:53.055069    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8646db9fbf-x72lb"]
Sep 18 17:52:53.057094 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:52:53.057063    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod719fcb78_5c8d_4a1f_811b_2d899e76c7b1.slice/crio-d99865fbfae168e56ccf09491e66a790d6074bf6a141ad87ecbff50eaab3bf3e WatchSource:0}: Error finding container d99865fbfae168e56ccf09491e66a790d6074bf6a141ad87ecbff50eaab3bf3e: Status 404 returned error can't find the container with id d99865fbfae168e56ccf09491e66a790d6074bf6a141ad87ecbff50eaab3bf3e
Sep 18 17:52:54.002452 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:54.002413    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb" event={"ID":"719fcb78-5c8d-4a1f-811b-2d899e76c7b1","Type":"ContainerStarted","Data":"d99865fbfae168e56ccf09491e66a790d6074bf6a141ad87ecbff50eaab3bf3e"}
Sep 18 17:52:56.012839 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.012803    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb" event={"ID":"719fcb78-5c8d-4a1f-811b-2d899e76c7b1","Type":"ContainerStarted","Data":"901302b313ab03e9fa6dba2c8973064d06e79f7945222081631471dd53c7b888"}
Sep 18 17:52:56.013262 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.012858    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb" event={"ID":"719fcb78-5c8d-4a1f-811b-2d899e76c7b1","Type":"ContainerStarted","Data":"5bf88a82a9325e6cc95e7d46ab51a7e72aac87fbfacb097ea1137ad9ce71ce96"}
Sep 18 17:52:56.013262 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.012877    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb" event={"ID":"719fcb78-5c8d-4a1f-811b-2d899e76c7b1","Type":"ContainerStarted","Data":"ece9abab9e75e965e985bacb43ae15dea9828c4b44eca6080a8ecb9f7d1678de"}
Sep 18 17:52:56.294913 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.294817    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-8646db9fbf-x72lb" podStartSLOduration=2.334270773 podStartE2EDuration="4.294799362s" totalImagesPullingTime="1.960528589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:52:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:52:56.070873697 +0000 UTC m=+180.148677606" watchObservedRunningTime="2026-09-18 17:52:56.294799362 +0000 UTC m=+180.372603261"
Sep 18 17:52:56.295715 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.295691    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-544f95d7cd-nm4wn"]
Sep 18 17:52:56.298002 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.297982    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.340338 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.340309    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-544f95d7cd-nm4wn"]
Sep 18 17:52:56.375369 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.375343    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-trusted-ca-bundle\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.375484 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.375373    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/abe4fff8-d507-4932-abec-e9c8b97454ff-console-oauth-config\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.375484 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.375403    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/abe4fff8-d507-4932-abec-e9c8b97454ff-console-serving-cert\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.375484 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.375473    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrpnm\" (UniqueName: \"kubernetes.io/projected/abe4fff8-d507-4932-abec-e9c8b97454ff-kube-api-access-mrpnm\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.375588 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.375525    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-service-ca\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.375588 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.375548    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-oauth-serving-cert\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.375588 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.375576    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-console-config\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.476853 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.476812    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/abe4fff8-d507-4932-abec-e9c8b97454ff-console-serving-cert\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.477016 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.476865    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrpnm\" (UniqueName: \"kubernetes.io/projected/abe4fff8-d507-4932-abec-e9c8b97454ff-kube-api-access-mrpnm\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.477016 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.476898    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-service-ca\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.477131 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.477057    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-oauth-serving-cert\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.477131 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.477120    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-console-config\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.477268 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.477166    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-trusted-ca-bundle\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.477318 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.477285    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/abe4fff8-d507-4932-abec-e9c8b97454ff-console-oauth-config\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.477708 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.477686    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-service-ca\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.477896 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.477868    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-oauth-serving-cert\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.478076 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.478057    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-trusted-ca-bundle\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.478262 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.478240    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-console-config\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.479524 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.479501    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/abe4fff8-d507-4932-abec-e9c8b97454ff-console-serving-cert\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.479704 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.479688    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/abe4fff8-d507-4932-abec-e9c8b97454ff-console-oauth-config\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.491744 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.491720    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrpnm\" (UniqueName: \"kubernetes.io/projected/abe4fff8-d507-4932-abec-e9c8b97454ff-kube-api-access-mrpnm\") pod \"console-544f95d7cd-nm4wn\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") " pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.607535 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.607456    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:52:56.890692 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:56.890620    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-544f95d7cd-nm4wn"]
Sep 18 17:52:56.900148 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:52:56.900111    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabe4fff8_d507_4932_abec_e9c8b97454ff.slice/crio-a59de77668320da964bda60a41997668b6bf8067129d83d7f8e8701a6b491696 WatchSource:0}: Error finding container a59de77668320da964bda60a41997668b6bf8067129d83d7f8e8701a6b491696: Status 404 returned error can't find the container with id a59de77668320da964bda60a41997668b6bf8067129d83d7f8e8701a6b491696
Sep 18 17:52:57.019298 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:57.019267    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-544f95d7cd-nm4wn" event={"ID":"abe4fff8-d507-4932-abec-e9c8b97454ff","Type":"ContainerStarted","Data":"7f99d15a4442db98e7e640333b82e038d5fae62d0240b4286cfc2905d92d4475"}
Sep 18 17:52:57.019690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:57.019314    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-544f95d7cd-nm4wn" event={"ID":"abe4fff8-d507-4932-abec-e9c8b97454ff","Type":"ContainerStarted","Data":"a59de77668320da964bda60a41997668b6bf8067129d83d7f8e8701a6b491696"}
Sep 18 17:52:57.063854 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:52:57.063803    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-544f95d7cd-nm4wn" podStartSLOduration=1.063785937 podStartE2EDuration="1.063785937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:52:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:52:57.062940197 +0000 UTC m=+181.140744100" watchObservedRunningTime="2026-09-18 17:52:57.063785937 +0000 UTC m=+181.141589887"
Sep 18 17:53:06.608092 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:06.608058    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:53:06.608464 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:06.608117    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:53:06.614061 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:06.614031    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:53:07.058656 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:07.058623    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:53:07.182122 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:07.182092    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cd7b65677-k2mlc"]
Sep 18 17:53:32.200030 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.199993    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6cd7b65677-k2mlc" podUID="0d51b6d5-780b-40b5-9c92-6eb83cafceb3" containerName="console" containerID="cri-o://8a0198e2817fe875b751026901a38e257f4f9d2ea08ce079f791a7f352cde002" gracePeriod=15
Sep 18 17:53:32.448226 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.448204    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:53:32.572072 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.571999    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-oauth-config\") pod \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") "
Sep 18 17:53:32.572072 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.572035    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-serving-cert\") pod \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") "
Sep 18 17:53:32.572072 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.572054    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-config\") pod \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") "
Sep 18 17:53:32.572072 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.572075    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-trusted-ca-bundle\") pod \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") "
Sep 18 17:53:32.572413 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.572116    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-service-ca\") pod \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") "
Sep 18 17:53:32.572413 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.572143    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnhp6\" (UniqueName: \"kubernetes.io/projected/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-kube-api-access-rnhp6\") pod \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") "
Sep 18 17:53:32.572413 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.572160    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-oauth-serving-cert\") pod \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\" (UID: \"0d51b6d5-780b-40b5-9c92-6eb83cafceb3\") "
Sep 18 17:53:32.572578 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.572552    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-config" (OuterVolumeSpecName: "console-config") pod "0d51b6d5-780b-40b5-9c92-6eb83cafceb3" (UID: "0d51b6d5-780b-40b5-9c92-6eb83cafceb3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:53:32.572694 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.572666    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-service-ca" (OuterVolumeSpecName: "service-ca") pod "0d51b6d5-780b-40b5-9c92-6eb83cafceb3" (UID: "0d51b6d5-780b-40b5-9c92-6eb83cafceb3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:53:32.572763 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.572690    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0d51b6d5-780b-40b5-9c92-6eb83cafceb3" (UID: "0d51b6d5-780b-40b5-9c92-6eb83cafceb3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:53:32.572763 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.572704    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0d51b6d5-780b-40b5-9c92-6eb83cafceb3" (UID: "0d51b6d5-780b-40b5-9c92-6eb83cafceb3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:53:32.574343 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.574318    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-kube-api-access-rnhp6" (OuterVolumeSpecName: "kube-api-access-rnhp6") pod "0d51b6d5-780b-40b5-9c92-6eb83cafceb3" (UID: "0d51b6d5-780b-40b5-9c92-6eb83cafceb3"). InnerVolumeSpecName "kube-api-access-rnhp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 17:53:32.574343 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.574322    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0d51b6d5-780b-40b5-9c92-6eb83cafceb3" (UID: "0d51b6d5-780b-40b5-9c92-6eb83cafceb3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:53:32.574481 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.574350    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0d51b6d5-780b-40b5-9c92-6eb83cafceb3" (UID: "0d51b6d5-780b-40b5-9c92-6eb83cafceb3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:53:32.673416 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.673377    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-trusted-ca-bundle\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:53:32.673416 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.673408    2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:53:32.673416 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.673419    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnhp6\" (UniqueName: \"kubernetes.io/projected/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-kube-api-access-rnhp6\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:53:32.673634 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.673429    2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-oauth-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:53:32.673634 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.673438    2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-oauth-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:53:32.673634 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.673448    2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:53:32.673634 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:32.673457    2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d51b6d5-780b-40b5-9c92-6eb83cafceb3-console-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:53:33.151598 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:33.151562    2568 generic.go:356] "Generic (PLEG): container finished" podID="0d51b6d5-780b-40b5-9c92-6eb83cafceb3" containerID="8a0198e2817fe875b751026901a38e257f4f9d2ea08ce079f791a7f352cde002" exitCode=0
Sep 18 17:53:33.151757 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:33.151603    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd7b65677-k2mlc"
Sep 18 17:53:33.151757 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:33.151654    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd7b65677-k2mlc" event={"ID":"0d51b6d5-780b-40b5-9c92-6eb83cafceb3","Type":"ContainerDied","Data":"8a0198e2817fe875b751026901a38e257f4f9d2ea08ce079f791a7f352cde002"}
Sep 18 17:53:33.151757 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:33.151696    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd7b65677-k2mlc" event={"ID":"0d51b6d5-780b-40b5-9c92-6eb83cafceb3","Type":"ContainerDied","Data":"b4b4a15a7d7701261f8caec69b5b40552ab693993274d7f82d3186d237f3f868"}
Sep 18 17:53:33.151757 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:33.151714    2568 scope.go:117] "RemoveContainer" containerID="8a0198e2817fe875b751026901a38e257f4f9d2ea08ce079f791a7f352cde002"
Sep 18 17:53:33.159903 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:33.159885    2568 scope.go:117] "RemoveContainer" containerID="8a0198e2817fe875b751026901a38e257f4f9d2ea08ce079f791a7f352cde002"
Sep 18 17:53:33.160161 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:53:33.160144    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a0198e2817fe875b751026901a38e257f4f9d2ea08ce079f791a7f352cde002\": container with ID starting with 8a0198e2817fe875b751026901a38e257f4f9d2ea08ce079f791a7f352cde002 not found: ID does not exist" containerID="8a0198e2817fe875b751026901a38e257f4f9d2ea08ce079f791a7f352cde002"
Sep 18 17:53:33.160237 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:33.160169    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a0198e2817fe875b751026901a38e257f4f9d2ea08ce079f791a7f352cde002"} err="failed to get container status \"8a0198e2817fe875b751026901a38e257f4f9d2ea08ce079f791a7f352cde002\": rpc error: code = NotFound desc = could not find container \"8a0198e2817fe875b751026901a38e257f4f9d2ea08ce079f791a7f352cde002\": container with ID starting with 8a0198e2817fe875b751026901a38e257f4f9d2ea08ce079f791a7f352cde002 not found: ID does not exist"
Sep 18 17:53:33.172214 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:33.172189    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cd7b65677-k2mlc"]
Sep 18 17:53:33.180302 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:33.180283    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cd7b65677-k2mlc"]
Sep 18 17:53:34.044418 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:53:34.044388    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d51b6d5-780b-40b5-9c92-6eb83cafceb3" path="/var/lib/kubelet/pods/0d51b6d5-780b-40b5-9c92-6eb83cafceb3/volumes"
Sep 18 17:54:02.962720 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:02.962670    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5ff77d5444-vgjtt"]
Sep 18 17:54:02.963277 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:02.963118    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d51b6d5-780b-40b5-9c92-6eb83cafceb3" containerName="console"
Sep 18 17:54:02.963277 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:02.963139    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d51b6d5-780b-40b5-9c92-6eb83cafceb3" containerName="console"
Sep 18 17:54:02.963277 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:02.963252    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d51b6d5-780b-40b5-9c92-6eb83cafceb3" containerName="console"
Sep 18 17:54:02.967655 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:02.967636    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:02.978118 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:02.978093    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5ff77d5444-vgjtt"]
Sep 18 17:54:02.989962 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:02.989932    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-serving-cert\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:02.990094 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:02.989974    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-config\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:02.990094 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:02.989996    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-oauth-serving-cert\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:02.990245 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:02.990088    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wclf2\" (UniqueName: \"kubernetes.io/projected/6ecacb4a-b719-4db2-8d24-7a41a88e5058-kube-api-access-wclf2\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:02.990245 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:02.990145    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-oauth-config\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:02.990245 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:02.990213    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-trusted-ca-bundle\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:02.990375 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:02.990244    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-service-ca\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.091408 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.091371    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-serving-cert\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.091611 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.091420    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-config\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.091611 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.091444    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-oauth-serving-cert\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.091611 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.091467    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wclf2\" (UniqueName: \"kubernetes.io/projected/6ecacb4a-b719-4db2-8d24-7a41a88e5058-kube-api-access-wclf2\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.091611 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.091501    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-oauth-config\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.091611 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.091525    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-trusted-ca-bundle\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.091611 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.091545    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-service-ca\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.092279 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.092254    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-config\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.092421 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.092347    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-service-ca\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.092487 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.092355    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-oauth-serving-cert\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.092525 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.092486    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-trusted-ca-bundle\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.093875 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.093857    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-oauth-config\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.094006 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.093986    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-serving-cert\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.100867 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.100847    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wclf2\" (UniqueName: \"kubernetes.io/projected/6ecacb4a-b719-4db2-8d24-7a41a88e5058-kube-api-access-wclf2\") pod \"console-5ff77d5444-vgjtt\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") " pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.277221 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.277112    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:03.548054 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:03.548026    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5ff77d5444-vgjtt"]
Sep 18 17:54:03.550523 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:54:03.550497    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ecacb4a_b719_4db2_8d24_7a41a88e5058.slice/crio-5ba2bd319077133c7f97562fd524b45264af54c195d29cd62a71465c27b8ecfd WatchSource:0}: Error finding container 5ba2bd319077133c7f97562fd524b45264af54c195d29cd62a71465c27b8ecfd: Status 404 returned error can't find the container with id 5ba2bd319077133c7f97562fd524b45264af54c195d29cd62a71465c27b8ecfd
Sep 18 17:54:04.276552 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:04.276510    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ff77d5444-vgjtt" event={"ID":"6ecacb4a-b719-4db2-8d24-7a41a88e5058","Type":"ContainerStarted","Data":"21ac068b1babf74cbd587d6bd1479a7afc416aecedb640d96aa70d25136dea90"}
Sep 18 17:54:04.276986 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:04.276567    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ff77d5444-vgjtt" event={"ID":"6ecacb4a-b719-4db2-8d24-7a41a88e5058","Type":"ContainerStarted","Data":"5ba2bd319077133c7f97562fd524b45264af54c195d29cd62a71465c27b8ecfd"}
Sep 18 17:54:04.296223 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:04.296164    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5ff77d5444-vgjtt" podStartSLOduration=2.296150574 podStartE2EDuration="2.296150574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:54:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:54:04.294730963 +0000 UTC m=+248.372534862" watchObservedRunningTime="2026-09-18 17:54:04.296150574 +0000 UTC m=+248.373954553"
Sep 18 17:54:13.277423 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:13.277314    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:13.277423 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:13.277390    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:13.282194 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:13.282157    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:13.312430 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:13.312398    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:54:13.409626 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:13.409596    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-544f95d7cd-nm4wn"]
Sep 18 17:54:38.427842 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.427797    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-544f95d7cd-nm4wn" podUID="abe4fff8-d507-4932-abec-e9c8b97454ff" containerName="console" containerID="cri-o://7f99d15a4442db98e7e640333b82e038d5fae62d0240b4286cfc2905d92d4475" gracePeriod=15
Sep 18 17:54:38.664773 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.664748    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:54:38.781053 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.780965    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-service-ca\") pod \"abe4fff8-d507-4932-abec-e9c8b97454ff\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") "
Sep 18 17:54:38.781053 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.781023    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/abe4fff8-d507-4932-abec-e9c8b97454ff-console-oauth-config\") pod \"abe4fff8-d507-4932-abec-e9c8b97454ff\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") "
Sep 18 17:54:38.781053 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.781042    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/abe4fff8-d507-4932-abec-e9c8b97454ff-console-serving-cert\") pod \"abe4fff8-d507-4932-abec-e9c8b97454ff\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") "
Sep 18 17:54:38.781340 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.781063    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-oauth-serving-cert\") pod \"abe4fff8-d507-4932-abec-e9c8b97454ff\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") "
Sep 18 17:54:38.781340 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.781109    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrpnm\" (UniqueName: \"kubernetes.io/projected/abe4fff8-d507-4932-abec-e9c8b97454ff-kube-api-access-mrpnm\") pod \"abe4fff8-d507-4932-abec-e9c8b97454ff\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") "
Sep 18 17:54:38.781340 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.781134    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-console-config\") pod \"abe4fff8-d507-4932-abec-e9c8b97454ff\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") "
Sep 18 17:54:38.781340 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.781203    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-trusted-ca-bundle\") pod \"abe4fff8-d507-4932-abec-e9c8b97454ff\" (UID: \"abe4fff8-d507-4932-abec-e9c8b97454ff\") "
Sep 18 17:54:38.781545 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.781397    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-service-ca" (OuterVolumeSpecName: "service-ca") pod "abe4fff8-d507-4932-abec-e9c8b97454ff" (UID: "abe4fff8-d507-4932-abec-e9c8b97454ff"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:54:38.781596 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.781544    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "abe4fff8-d507-4932-abec-e9c8b97454ff" (UID: "abe4fff8-d507-4932-abec-e9c8b97454ff"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:54:38.781809 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.781778    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-console-config" (OuterVolumeSpecName: "console-config") pod "abe4fff8-d507-4932-abec-e9c8b97454ff" (UID: "abe4fff8-d507-4932-abec-e9c8b97454ff"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:54:38.781968 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.781886    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "abe4fff8-d507-4932-abec-e9c8b97454ff" (UID: "abe4fff8-d507-4932-abec-e9c8b97454ff"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:54:38.783348 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.783323    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abe4fff8-d507-4932-abec-e9c8b97454ff-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "abe4fff8-d507-4932-abec-e9c8b97454ff" (UID: "abe4fff8-d507-4932-abec-e9c8b97454ff"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:54:38.783451 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.783355    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abe4fff8-d507-4932-abec-e9c8b97454ff-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "abe4fff8-d507-4932-abec-e9c8b97454ff" (UID: "abe4fff8-d507-4932-abec-e9c8b97454ff"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:54:38.783451 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.783414    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abe4fff8-d507-4932-abec-e9c8b97454ff-kube-api-access-mrpnm" (OuterVolumeSpecName: "kube-api-access-mrpnm") pod "abe4fff8-d507-4932-abec-e9c8b97454ff" (UID: "abe4fff8-d507-4932-abec-e9c8b97454ff"). InnerVolumeSpecName "kube-api-access-mrpnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 17:54:38.881971 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.881940    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-trusted-ca-bundle\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:54:38.881971 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.881968    2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:54:38.882146 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.881982    2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/abe4fff8-d507-4932-abec-e9c8b97454ff-console-oauth-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:54:38.882146 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.881994    2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/abe4fff8-d507-4932-abec-e9c8b97454ff-console-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:54:38.882146 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.882006    2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-oauth-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:54:38.882146 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.882017    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrpnm\" (UniqueName: \"kubernetes.io/projected/abe4fff8-d507-4932-abec-e9c8b97454ff-kube-api-access-mrpnm\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:54:38.882146 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:38.882028    2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/abe4fff8-d507-4932-abec-e9c8b97454ff-console-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:54:39.406376 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:39.406344    2568 generic.go:356] "Generic (PLEG): container finished" podID="abe4fff8-d507-4932-abec-e9c8b97454ff" containerID="7f99d15a4442db98e7e640333b82e038d5fae62d0240b4286cfc2905d92d4475" exitCode=0
Sep 18 17:54:39.406548 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:39.406383    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-544f95d7cd-nm4wn"
Sep 18 17:54:39.406548 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:39.406431    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-544f95d7cd-nm4wn" event={"ID":"abe4fff8-d507-4932-abec-e9c8b97454ff","Type":"ContainerDied","Data":"7f99d15a4442db98e7e640333b82e038d5fae62d0240b4286cfc2905d92d4475"}
Sep 18 17:54:39.406548 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:39.406477    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-544f95d7cd-nm4wn" event={"ID":"abe4fff8-d507-4932-abec-e9c8b97454ff","Type":"ContainerDied","Data":"a59de77668320da964bda60a41997668b6bf8067129d83d7f8e8701a6b491696"}
Sep 18 17:54:39.406548 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:39.406496    2568 scope.go:117] "RemoveContainer" containerID="7f99d15a4442db98e7e640333b82e038d5fae62d0240b4286cfc2905d92d4475"
Sep 18 17:54:39.415127 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:39.415109    2568 scope.go:117] "RemoveContainer" containerID="7f99d15a4442db98e7e640333b82e038d5fae62d0240b4286cfc2905d92d4475"
Sep 18 17:54:39.415377 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:54:39.415359    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f99d15a4442db98e7e640333b82e038d5fae62d0240b4286cfc2905d92d4475\": container with ID starting with 7f99d15a4442db98e7e640333b82e038d5fae62d0240b4286cfc2905d92d4475 not found: ID does not exist" containerID="7f99d15a4442db98e7e640333b82e038d5fae62d0240b4286cfc2905d92d4475"
Sep 18 17:54:39.415456 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:39.415384    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f99d15a4442db98e7e640333b82e038d5fae62d0240b4286cfc2905d92d4475"} err="failed to get container status \"7f99d15a4442db98e7e640333b82e038d5fae62d0240b4286cfc2905d92d4475\": rpc error: code = NotFound desc = could not find container \"7f99d15a4442db98e7e640333b82e038d5fae62d0240b4286cfc2905d92d4475\": container with ID starting with 7f99d15a4442db98e7e640333b82e038d5fae62d0240b4286cfc2905d92d4475 not found: ID does not exist"
Sep 18 17:54:39.429082 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:39.429057    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-544f95d7cd-nm4wn"]
Sep 18 17:54:39.431698 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:39.431678    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-544f95d7cd-nm4wn"]
Sep 18 17:54:40.043581 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:40.043552    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abe4fff8-d507-4932-abec-e9c8b97454ff" path="/var/lib/kubelet/pods/abe4fff8-d507-4932-abec-e9c8b97454ff/volumes"
Sep 18 17:54:57.949199 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:57.949145    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 17:54:57.950995 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:57.950966    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 17:54:57.952350 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:54:57.952323    2568 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 17:55:40.422777 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:40.422740    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hz8rj"]
Sep 18 17:55:40.423348 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:40.423215    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abe4fff8-d507-4932-abec-e9c8b97454ff" containerName="console"
Sep 18 17:55:40.423348 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:40.423234    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="abe4fff8-d507-4932-abec-e9c8b97454ff" containerName="console"
Sep 18 17:55:40.423348 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:40.423329    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="abe4fff8-d507-4932-abec-e9c8b97454ff" containerName="console"
Sep 18 17:55:40.426393 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:40.426339    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hz8rj"
Sep 18 17:55:40.430804 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:40.430775    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 18 17:55:40.431921 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:40.431894    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 18 17:55:40.444153 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:40.444126    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hz8rj"]
Sep 18 17:55:40.592150 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:40.592068    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxgx2\" (UniqueName: \"kubernetes.io/projected/626d2916-5398-44f0-a704-df370813c007-kube-api-access-kxgx2\") pod \"cert-manager-cainjector-559cdd8c87-hz8rj\" (UID: \"626d2916-5398-44f0-a704-df370813c007\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hz8rj"
Sep 18 17:55:40.592332 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:40.592212    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/626d2916-5398-44f0-a704-df370813c007-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hz8rj\" (UID: \"626d2916-5398-44f0-a704-df370813c007\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hz8rj"
Sep 18 17:55:40.692979 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:40.692883    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/626d2916-5398-44f0-a704-df370813c007-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hz8rj\" (UID: \"626d2916-5398-44f0-a704-df370813c007\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hz8rj"
Sep 18 17:55:40.692979 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:40.692930    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxgx2\" (UniqueName: \"kubernetes.io/projected/626d2916-5398-44f0-a704-df370813c007-kube-api-access-kxgx2\") pod \"cert-manager-cainjector-559cdd8c87-hz8rj\" (UID: \"626d2916-5398-44f0-a704-df370813c007\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hz8rj"
Sep 18 17:55:40.701887 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:40.701857    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxgx2\" (UniqueName: \"kubernetes.io/projected/626d2916-5398-44f0-a704-df370813c007-kube-api-access-kxgx2\") pod \"cert-manager-cainjector-559cdd8c87-hz8rj\" (UID: \"626d2916-5398-44f0-a704-df370813c007\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hz8rj"
Sep 18 17:55:40.702156 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:40.702139    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/626d2916-5398-44f0-a704-df370813c007-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hz8rj\" (UID: \"626d2916-5398-44f0-a704-df370813c007\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hz8rj"
Sep 18 17:55:40.745244 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:40.745197    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hz8rj"
Sep 18 17:55:41.029407 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:41.029329    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hz8rj"]
Sep 18 17:55:41.033982 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:55:41.033951    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod626d2916_5398_44f0_a704_df370813c007.slice/crio-14c14b8535ea00b008c4737fa55f30e058a46ba362fee59aa1c00b897bfd38e0 WatchSource:0}: Error finding container 14c14b8535ea00b008c4737fa55f30e058a46ba362fee59aa1c00b897bfd38e0: Status 404 returned error can't find the container with id 14c14b8535ea00b008c4737fa55f30e058a46ba362fee59aa1c00b897bfd38e0
Sep 18 17:55:41.036280 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:41.036264    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 17:55:41.651110 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:41.651076    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hz8rj" event={"ID":"626d2916-5398-44f0-a704-df370813c007","Type":"ContainerStarted","Data":"14c14b8535ea00b008c4737fa55f30e058a46ba362fee59aa1c00b897bfd38e0"}
Sep 18 17:55:44.668576 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:44.668542    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hz8rj" event={"ID":"626d2916-5398-44f0-a704-df370813c007","Type":"ContainerStarted","Data":"f40fe69c9f3629482f20d802643d85be25a1f99dfb88f4c632723e2b9ad02fca"}
Sep 18 17:55:44.686507 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:44.686452    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hz8rj" podStartSLOduration=1.365036001 podStartE2EDuration="4.686435149s" totalImagesPullingTime="3.321399148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:55:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:55:44.685232211 +0000 UTC m=+348.763036111" watchObservedRunningTime="2026-09-18 17:55:44.686435149 +0000 UTC m=+348.764239049"
Sep 18 17:55:49.244106 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:49.244069    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zwrm5"]
Sep 18 17:55:49.247257 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:49.247239    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-zwrm5"
Sep 18 17:55:49.251016 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:49.250990    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-lmzc4\""
Sep 18 17:55:49.264839 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:49.264811    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75jwg\" (UniqueName: \"kubernetes.io/projected/84e85a5a-073f-48ea-80f6-67a28232309a-kube-api-access-75jwg\") pod \"cert-manager-55b4d954d5-zwrm5\" (UID: \"84e85a5a-073f-48ea-80f6-67a28232309a\") " pod="cert-manager/cert-manager-55b4d954d5-zwrm5"
Sep 18 17:55:49.264986 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:49.264858    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84e85a5a-073f-48ea-80f6-67a28232309a-bound-sa-token\") pod \"cert-manager-55b4d954d5-zwrm5\" (UID: \"84e85a5a-073f-48ea-80f6-67a28232309a\") " pod="cert-manager/cert-manager-55b4d954d5-zwrm5"
Sep 18 17:55:49.269590 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:49.269561    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zwrm5"]
Sep 18 17:55:49.365948 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:49.365904    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75jwg\" (UniqueName: \"kubernetes.io/projected/84e85a5a-073f-48ea-80f6-67a28232309a-kube-api-access-75jwg\") pod \"cert-manager-55b4d954d5-zwrm5\" (UID: \"84e85a5a-073f-48ea-80f6-67a28232309a\") " pod="cert-manager/cert-manager-55b4d954d5-zwrm5"
Sep 18 17:55:49.366214 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:49.365965    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84e85a5a-073f-48ea-80f6-67a28232309a-bound-sa-token\") pod \"cert-manager-55b4d954d5-zwrm5\" (UID: \"84e85a5a-073f-48ea-80f6-67a28232309a\") " pod="cert-manager/cert-manager-55b4d954d5-zwrm5"
Sep 18 17:55:49.375809 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:49.375774    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84e85a5a-073f-48ea-80f6-67a28232309a-bound-sa-token\") pod \"cert-manager-55b4d954d5-zwrm5\" (UID: \"84e85a5a-073f-48ea-80f6-67a28232309a\") " pod="cert-manager/cert-manager-55b4d954d5-zwrm5"
Sep 18 17:55:49.376082 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:49.376059    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75jwg\" (UniqueName: \"kubernetes.io/projected/84e85a5a-073f-48ea-80f6-67a28232309a-kube-api-access-75jwg\") pod \"cert-manager-55b4d954d5-zwrm5\" (UID: \"84e85a5a-073f-48ea-80f6-67a28232309a\") " pod="cert-manager/cert-manager-55b4d954d5-zwrm5"
Sep 18 17:55:49.555944 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:49.555854    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-zwrm5"
Sep 18 17:55:49.830645 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:49.830618    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zwrm5"]
Sep 18 17:55:49.833324 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:55:49.833293    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84e85a5a_073f_48ea_80f6_67a28232309a.slice/crio-cb90adcb9c282b3e4e1a6833d98d235b42404447c8fb5d68e34ee539da69ea4b WatchSource:0}: Error finding container cb90adcb9c282b3e4e1a6833d98d235b42404447c8fb5d68e34ee539da69ea4b: Status 404 returned error can't find the container with id cb90adcb9c282b3e4e1a6833d98d235b42404447c8fb5d68e34ee539da69ea4b
Sep 18 17:55:50.006988 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.006951    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfbmp"]
Sep 18 17:55:50.010163 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.010145    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfbmp"
Sep 18 17:55:50.013165 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.013134    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 18 17:55:50.013580 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.013346    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-x87sg\""
Sep 18 17:55:50.013580 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.013482    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 18 17:55:50.023423 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.023398    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfbmp"]
Sep 18 17:55:50.070400 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.070358    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9d303be-d7c5-4ec2-87cb-318bb40b5e46-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tfbmp\" (UID: \"a9d303be-d7c5-4ec2-87cb-318bb40b5e46\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfbmp"
Sep 18 17:55:50.070611 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.070452    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8j2k\" (UniqueName: \"kubernetes.io/projected/a9d303be-d7c5-4ec2-87cb-318bb40b5e46-kube-api-access-v8j2k\") pod \"openshift-lws-operator-fd8ccff4c-tfbmp\" (UID: \"a9d303be-d7c5-4ec2-87cb-318bb40b5e46\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfbmp"
Sep 18 17:55:50.170832 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.170797    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8j2k\" (UniqueName: \"kubernetes.io/projected/a9d303be-d7c5-4ec2-87cb-318bb40b5e46-kube-api-access-v8j2k\") pod \"openshift-lws-operator-fd8ccff4c-tfbmp\" (UID: \"a9d303be-d7c5-4ec2-87cb-318bb40b5e46\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfbmp"
Sep 18 17:55:50.171049 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.170874    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9d303be-d7c5-4ec2-87cb-318bb40b5e46-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tfbmp\" (UID: \"a9d303be-d7c5-4ec2-87cb-318bb40b5e46\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfbmp"
Sep 18 17:55:50.171268 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.171250    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9d303be-d7c5-4ec2-87cb-318bb40b5e46-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tfbmp\" (UID: \"a9d303be-d7c5-4ec2-87cb-318bb40b5e46\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfbmp"
Sep 18 17:55:50.180859 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.180827    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8j2k\" (UniqueName: \"kubernetes.io/projected/a9d303be-d7c5-4ec2-87cb-318bb40b5e46-kube-api-access-v8j2k\") pod \"openshift-lws-operator-fd8ccff4c-tfbmp\" (UID: \"a9d303be-d7c5-4ec2-87cb-318bb40b5e46\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfbmp"
Sep 18 17:55:50.323495 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.323454    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfbmp"
Sep 18 17:55:50.607770 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.607744    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfbmp"]
Sep 18 17:55:50.610051 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:55:50.610014    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9d303be_d7c5_4ec2_87cb_318bb40b5e46.slice/crio-489365c5ae7b855e879d5ca855c529278ba6c5832a77235b1e61cab16c6e935c WatchSource:0}: Error finding container 489365c5ae7b855e879d5ca855c529278ba6c5832a77235b1e61cab16c6e935c: Status 404 returned error can't find the container with id 489365c5ae7b855e879d5ca855c529278ba6c5832a77235b1e61cab16c6e935c
Sep 18 17:55:50.694023 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.693971    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfbmp" event={"ID":"a9d303be-d7c5-4ec2-87cb-318bb40b5e46","Type":"ContainerStarted","Data":"489365c5ae7b855e879d5ca855c529278ba6c5832a77235b1e61cab16c6e935c"}
Sep 18 17:55:50.695526 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.695502    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-zwrm5" event={"ID":"84e85a5a-073f-48ea-80f6-67a28232309a","Type":"ContainerStarted","Data":"b9fbc19365bf627a4a08ff106317d5125d8a5ffd8f183aa39adf5655aca01f7b"}
Sep 18 17:55:50.695655 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.695537    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-zwrm5" event={"ID":"84e85a5a-073f-48ea-80f6-67a28232309a","Type":"ContainerStarted","Data":"cb90adcb9c282b3e4e1a6833d98d235b42404447c8fb5d68e34ee539da69ea4b"}
Sep 18 17:55:50.720350 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:50.720297    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-zwrm5" podStartSLOduration=1.720279647 podStartE2EDuration="1.720279647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:55:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:55:50.719039844 +0000 UTC m=+354.796843786" watchObservedRunningTime="2026-09-18 17:55:50.720279647 +0000 UTC m=+354.798083545"
Sep 18 17:55:55.723805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:55.723767    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfbmp" event={"ID":"a9d303be-d7c5-4ec2-87cb-318bb40b5e46","Type":"ContainerStarted","Data":"1eecb4a1502a9e2eb6b3ff2200bf0231921c664d66492eb0ec89332757223831"}
Sep 18 17:55:55.743097 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:55:55.743035    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfbmp" podStartSLOduration=2.560189027 podStartE2EDuration="6.743017266s" totalImagesPullingTime="4.182828239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:55:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:55:55.741491675 +0000 UTC m=+359.819295574" watchObservedRunningTime="2026-09-18 17:55:55.743017266 +0000 UTC m=+359.820821168"
Sep 18 17:56:14.431018 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.430982    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"]
Sep 18 17:56:14.438288 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.438256    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:14.443971 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:56:14.443941    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-service-cert\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret"
Sep 18 17:56:14.444369 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.444316    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6jc6c\""
Sep 18 17:56:14.445012 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.444570    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 18 17:56:14.445012 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.444939    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 18 17:56:14.445116 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.445012    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 18 17:56:14.474505 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.474480    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"]
Sep 18 17:56:14.538632 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.538591    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-webhook-cert\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:14.538805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.538634    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-manifests\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:14.538805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.538705    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9ts9\" (UniqueName: \"kubernetes.io/projected/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-kube-api-access-n9ts9\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:14.538805 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.538755    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:14.538953 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.538868    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-tmp\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:14.639678 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.639641    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:14.639872 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.639698    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-tmp\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:14.639872 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.639733    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-webhook-cert\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:14.639872 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.639756    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-manifests\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:14.639872 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.639784    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9ts9\" (UniqueName: \"kubernetes.io/projected/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-kube-api-access-n9ts9\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:14.640256 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.640224    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-manifests\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:14.641965 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.641936    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-tmp\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:14.650388 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:14.650362    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9ts9\" (UniqueName: \"kubernetes.io/projected/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-kube-api-access-n9ts9\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:15.460586 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.460551    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bcc9bbb87-kkhlz"]
Sep 18 17:56:15.463807 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.463790    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.493431 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.493402    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bcc9bbb87-kkhlz"]
Sep 18 17:56:15.546715 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.546681    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98599823-a747-47b5-a3d7-a6a8a9fa6127-console-serving-cert\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.546872 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.546719    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98599823-a747-47b5-a3d7-a6a8a9fa6127-trusted-ca-bundle\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.546872 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.546751    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98599823-a747-47b5-a3d7-a6a8a9fa6127-oauth-serving-cert\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.546872 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.546805    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98599823-a747-47b5-a3d7-a6a8a9fa6127-service-ca\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.546872 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.546835    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlkl6\" (UniqueName: \"kubernetes.io/projected/98599823-a747-47b5-a3d7-a6a8a9fa6127-kube-api-access-dlkl6\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.547012 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.546932    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98599823-a747-47b5-a3d7-a6a8a9fa6127-console-config\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.547012 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.546961    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98599823-a747-47b5-a3d7-a6a8a9fa6127-console-oauth-config\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.640597 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:56:15.640556    2568 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition
Sep 18 17:56:15.640768 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:56:15.640650    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-webhook-cert podName:0c818f22-b7e4-4cc2-9fb9-58fba431acd8 nodeName:}" failed. No retries permitted until 2026-09-18 17:56:16.140632002 +0000 UTC m=+380.218435878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-webhook-cert") pod "opendatahub-operator-controller-manager-649dc7b766-xkzzk" (UID: "0c818f22-b7e4-4cc2-9fb9-58fba431acd8") : failed to sync secret cache: timed out waiting for the condition
Sep 18 17:56:15.640768 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:56:15.640556    2568 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition
Sep 18 17:56:15.640768 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:56:15.640710    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-apiservice-cert podName:0c818f22-b7e4-4cc2-9fb9-58fba431acd8 nodeName:}" failed. No retries permitted until 2026-09-18 17:56:16.14069808 +0000 UTC m=+380.218501970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "apiservice-cert" (UniqueName: "kubernetes.io/secret/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-apiservice-cert") pod "opendatahub-operator-controller-manager-649dc7b766-xkzzk" (UID: "0c818f22-b7e4-4cc2-9fb9-58fba431acd8") : failed to sync secret cache: timed out waiting for the condition
Sep 18 17:56:15.648029 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.647999    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98599823-a747-47b5-a3d7-a6a8a9fa6127-console-serving-cert\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.648136 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.648037    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98599823-a747-47b5-a3d7-a6a8a9fa6127-trusted-ca-bundle\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.648136 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.648053    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98599823-a747-47b5-a3d7-a6a8a9fa6127-oauth-serving-cert\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.648136 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.648081    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98599823-a747-47b5-a3d7-a6a8a9fa6127-service-ca\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.648136 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.648101    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlkl6\" (UniqueName: \"kubernetes.io/projected/98599823-a747-47b5-a3d7-a6a8a9fa6127-kube-api-access-dlkl6\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.648319 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.648199    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98599823-a747-47b5-a3d7-a6a8a9fa6127-console-config\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.648319 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.648252    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98599823-a747-47b5-a3d7-a6a8a9fa6127-console-oauth-config\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.648864 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.648839    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/98599823-a747-47b5-a3d7-a6a8a9fa6127-oauth-serving-cert\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.648964 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.648939    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/98599823-a747-47b5-a3d7-a6a8a9fa6127-service-ca\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.649029 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.649013    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98599823-a747-47b5-a3d7-a6a8a9fa6127-trusted-ca-bundle\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.649063 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.649032    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/98599823-a747-47b5-a3d7-a6a8a9fa6127-console-config\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.650562 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.650540    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/98599823-a747-47b5-a3d7-a6a8a9fa6127-console-oauth-config\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.650704 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.650687    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/98599823-a747-47b5-a3d7-a6a8a9fa6127-console-serving-cert\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.667310 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.667281    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlkl6\" (UniqueName: \"kubernetes.io/projected/98599823-a747-47b5-a3d7-a6a8a9fa6127-kube-api-access-dlkl6\") pod \"console-bcc9bbb87-kkhlz\" (UID: \"98599823-a747-47b5-a3d7-a6a8a9fa6127\") " pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.773714 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.773630    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:15.862324 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:15.862296    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 18 17:56:16.072579 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:16.072556    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bcc9bbb87-kkhlz"]
Sep 18 17:56:16.074385 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:56:16.074364    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98599823_a747_47b5_a3d7_a6a8a9fa6127.slice/crio-d4294619cf4327f1fc90a4d532d08e88e83924dccd2d959ebc8edce95a28596d WatchSource:0}: Error finding container d4294619cf4327f1fc90a4d532d08e88e83924dccd2d959ebc8edce95a28596d: Status 404 returned error can't find the container with id d4294619cf4327f1fc90a4d532d08e88e83924dccd2d959ebc8edce95a28596d
Sep 18 17:56:16.153741 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:16.153552    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-webhook-cert\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:16.153741 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:16.153641    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:16.156020 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:16.155998    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:16.156381 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:16.156364    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c818f22-b7e4-4cc2-9fb9-58fba431acd8-webhook-cert\") pod \"opendatahub-operator-controller-manager-649dc7b766-xkzzk\" (UID: \"0c818f22-b7e4-4cc2-9fb9-58fba431acd8\") " pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:16.248346 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:16.248308    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:16.550992 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:16.550959    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"]
Sep 18 17:56:16.554866 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:56:16.554839    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c818f22_b7e4_4cc2_9fb9_58fba431acd8.slice/crio-de8b2c1949c56721725bf349f545e9481372a3a25f862263f166fdf904d1edb8 WatchSource:0}: Error finding container de8b2c1949c56721725bf349f545e9481372a3a25f862263f166fdf904d1edb8: Status 404 returned error can't find the container with id de8b2c1949c56721725bf349f545e9481372a3a25f862263f166fdf904d1edb8
Sep 18 17:56:16.809544 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:16.809455    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk" event={"ID":"0c818f22-b7e4-4cc2-9fb9-58fba431acd8","Type":"ContainerStarted","Data":"de8b2c1949c56721725bf349f545e9481372a3a25f862263f166fdf904d1edb8"}
Sep 18 17:56:16.811010 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:16.810986    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bcc9bbb87-kkhlz" event={"ID":"98599823-a747-47b5-a3d7-a6a8a9fa6127","Type":"ContainerStarted","Data":"63ceb571d75be08ad7cd138c78ef953b215350251765619344b780448f7d8689"}
Sep 18 17:56:16.811155 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:16.811023    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bcc9bbb87-kkhlz" event={"ID":"98599823-a747-47b5-a3d7-a6a8a9fa6127","Type":"ContainerStarted","Data":"d4294619cf4327f1fc90a4d532d08e88e83924dccd2d959ebc8edce95a28596d"}
Sep 18 17:56:16.836363 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:16.836316    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bcc9bbb87-kkhlz" podStartSLOduration=1.8362989870000002 podStartE2EDuration="1.836298987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:56:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:56:16.836070912 +0000 UTC m=+380.913874813" watchObservedRunningTime="2026-09-18 17:56:16.836298987 +0000 UTC m=+380.914102883"
Sep 18 17:56:19.831811 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:19.831723    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk" event={"ID":"0c818f22-b7e4-4cc2-9fb9-58fba431acd8","Type":"ContainerStarted","Data":"e9fdab0d9fb6c888de74c4fa32dff168ec8e8a4052a1999b5d4585f6161de843"}
Sep 18 17:56:20.837413 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:20.837327    2568 generic.go:356] "Generic (PLEG): container finished" podID="0c818f22-b7e4-4cc2-9fb9-58fba431acd8" containerID="e9fdab0d9fb6c888de74c4fa32dff168ec8e8a4052a1999b5d4585f6161de843" exitCode=0
Sep 18 17:56:20.837413 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:20.837402    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk" event={"ID":"0c818f22-b7e4-4cc2-9fb9-58fba431acd8","Type":"ContainerDied","Data":"e9fdab0d9fb6c888de74c4fa32dff168ec8e8a4052a1999b5d4585f6161de843"}
Sep 18 17:56:21.843933 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:21.843899    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk" event={"ID":"0c818f22-b7e4-4cc2-9fb9-58fba431acd8","Type":"ContainerStarted","Data":"35a395168c18c7b6f1723d57b4a6aaaa4c6f412a6096c02b51412d893fdccdb5"}
Sep 18 17:56:21.844353 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:21.844056    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:21.890194 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:21.890135    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk" podStartSLOduration=3.543863172 podStartE2EDuration="7.890118419s" totalImagesPullingTime="3.273026781s" totalInitContainerRuntime="1.073228466s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:56:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:56:21.888989165 +0000 UTC m=+385.966793067" watchObservedRunningTime="2026-09-18 17:56:21.890118419 +0000 UTC m=+385.967922317"
Sep 18 17:56:25.774057 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:25.774014    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:25.774488 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:25.774089    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:25.778932 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:25.778903    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:25.866057 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:25.866029    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bcc9bbb87-kkhlz"
Sep 18 17:56:25.925776 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:25.925745    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5ff77d5444-vgjtt"]
Sep 18 17:56:32.850906 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:32.850872    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-649dc7b766-xkzzk"
Sep 18 17:56:35.486431 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:35.486394    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln"]
Sep 18 17:56:35.497172 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:35.497147    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln"
Sep 18 17:56:35.500371 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:35.500342    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 18 17:56:35.500914 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:35.500891    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 17:56:35.501373 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:35.501357    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 17:56:35.501456 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:35.501378    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 18 17:56:35.510472 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:35.510452    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln"]
Sep 18 17:56:35.631500 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:35.631463    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a06b09b-0510-4c27-8ee0-182b172f05b1-tmp\") pod \"kube-auth-proxy-6b7b7f7d54-5mfln\" (UID: \"6a06b09b-0510-4c27-8ee0-182b172f05b1\") " pod="openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln"
Sep 18 17:56:35.631500 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:35.631498    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68j52\" (UniqueName: \"kubernetes.io/projected/6a06b09b-0510-4c27-8ee0-182b172f05b1-kube-api-access-68j52\") pod \"kube-auth-proxy-6b7b7f7d54-5mfln\" (UID: \"6a06b09b-0510-4c27-8ee0-182b172f05b1\") " pod="openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln"
Sep 18 17:56:35.631726 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:35.631608    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a06b09b-0510-4c27-8ee0-182b172f05b1-tls-certs\") pod \"kube-auth-proxy-6b7b7f7d54-5mfln\" (UID: \"6a06b09b-0510-4c27-8ee0-182b172f05b1\") " pod="openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln"
Sep 18 17:56:35.732313 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:35.732272    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a06b09b-0510-4c27-8ee0-182b172f05b1-tmp\") pod \"kube-auth-proxy-6b7b7f7d54-5mfln\" (UID: \"6a06b09b-0510-4c27-8ee0-182b172f05b1\") " pod="openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln"
Sep 18 17:56:35.732313 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:35.732310    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68j52\" (UniqueName: \"kubernetes.io/projected/6a06b09b-0510-4c27-8ee0-182b172f05b1-kube-api-access-68j52\") pod \"kube-auth-proxy-6b7b7f7d54-5mfln\" (UID: \"6a06b09b-0510-4c27-8ee0-182b172f05b1\") " pod="openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln"
Sep 18 17:56:35.732543 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:35.732374    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a06b09b-0510-4c27-8ee0-182b172f05b1-tls-certs\") pod \"kube-auth-proxy-6b7b7f7d54-5mfln\" (UID: \"6a06b09b-0510-4c27-8ee0-182b172f05b1\") " pod="openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln"
Sep 18 17:56:35.732543 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:56:35.732475    2568 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 18 17:56:35.732543 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:56:35.732540    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a06b09b-0510-4c27-8ee0-182b172f05b1-tls-certs podName:6a06b09b-0510-4c27-8ee0-182b172f05b1 nodeName:}" failed. No retries permitted until 2026-09-18 17:56:36.232523054 +0000 UTC m=+400.310326930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6a06b09b-0510-4c27-8ee0-182b172f05b1-tls-certs") pod "kube-auth-proxy-6b7b7f7d54-5mfln" (UID: "6a06b09b-0510-4c27-8ee0-182b172f05b1") : secret "kube-auth-proxy-tls" not found
Sep 18 17:56:35.734697 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:35.734676    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a06b09b-0510-4c27-8ee0-182b172f05b1-tmp\") pod \"kube-auth-proxy-6b7b7f7d54-5mfln\" (UID: \"6a06b09b-0510-4c27-8ee0-182b172f05b1\") " pod="openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln"
Sep 18 17:56:35.767367 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:35.767296    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68j52\" (UniqueName: \"kubernetes.io/projected/6a06b09b-0510-4c27-8ee0-182b172f05b1-kube-api-access-68j52\") pod \"kube-auth-proxy-6b7b7f7d54-5mfln\" (UID: \"6a06b09b-0510-4c27-8ee0-182b172f05b1\") " pod="openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln"
Sep 18 17:56:36.238118 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:36.238080    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a06b09b-0510-4c27-8ee0-182b172f05b1-tls-certs\") pod \"kube-auth-proxy-6b7b7f7d54-5mfln\" (UID: \"6a06b09b-0510-4c27-8ee0-182b172f05b1\") " pod="openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln"
Sep 18 17:56:36.240642 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:36.240619    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a06b09b-0510-4c27-8ee0-182b172f05b1-tls-certs\") pod \"kube-auth-proxy-6b7b7f7d54-5mfln\" (UID: \"6a06b09b-0510-4c27-8ee0-182b172f05b1\") " pod="openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln"
Sep 18 17:56:36.408411 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:36.408371    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln"
Sep 18 17:56:36.699109 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:36.699066    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln"]
Sep 18 17:56:36.703965 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:56:36.703937    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a06b09b_0510_4c27_8ee0_182b172f05b1.slice/crio-58b2f5cb9bd6f80f25b3412f41f5259c553482e3710e3c99ce357eefdf689439 WatchSource:0}: Error finding container 58b2f5cb9bd6f80f25b3412f41f5259c553482e3710e3c99ce357eefdf689439: Status 404 returned error can't find the container with id 58b2f5cb9bd6f80f25b3412f41f5259c553482e3710e3c99ce357eefdf689439
Sep 18 17:56:36.911108 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:36.911072    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln" event={"ID":"6a06b09b-0510-4c27-8ee0-182b172f05b1","Type":"ContainerStarted","Data":"58b2f5cb9bd6f80f25b3412f41f5259c553482e3710e3c99ce357eefdf689439"}
Sep 18 17:56:38.141749 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.141717    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"]
Sep 18 17:56:38.145123 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.145105    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.147986 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.147961    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 18 17:56:38.149245 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.149219    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 17:56:38.149245 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.149309    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-m2bc8\""
Sep 18 17:56:38.149533 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.149396    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 18 17:56:38.150202 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.150152    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 18 17:56:38.163850 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.163824    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"]
Sep 18 17:56:38.258590 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.258553    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.258590 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.258596    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxbxg\" (UniqueName: \"kubernetes.io/projected/e451d3c6-52ec-418f-9229-8b9a3588ed3c-kube-api-access-zxbxg\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.258829 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.258644    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.258829 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.258695    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.258829 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.258771    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.258829 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.258805    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e451d3c6-52ec-418f-9229-8b9a3588ed3c-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.259010 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.258884    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.360079 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.360038    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.360286 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.360104    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.360286 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.360129    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxbxg\" (UniqueName: \"kubernetes.io/projected/e451d3c6-52ec-418f-9229-8b9a3588ed3c-kube-api-access-zxbxg\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.360286 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.360260    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.360477 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.360347    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.360477 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.360406    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.360477 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.360456    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e451d3c6-52ec-418f-9229-8b9a3588ed3c-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.361314 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.361255    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.363234 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.363210    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.363350 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.363250    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.363563 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.363453    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.363563 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.363521    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e451d3c6-52ec-418f-9229-8b9a3588ed3c-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.371857 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.371832    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxbxg\" (UniqueName: \"kubernetes.io/projected/e451d3c6-52ec-418f-9229-8b9a3588ed3c-kube-api-access-zxbxg\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.377264 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.377212    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-5m68v\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:38.458414 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:38.458379    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:39.806827 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:39.806801    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"]
Sep 18 17:56:39.808890 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:56:39.808866    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode451d3c6_52ec_418f_9229_8b9a3588ed3c.slice/crio-b926990f47a68be4b8de7f5b618bf47c9176976653dcbccf5c48e49c5b0cd6f1 WatchSource:0}: Error finding container b926990f47a68be4b8de7f5b618bf47c9176976653dcbccf5c48e49c5b0cd6f1: Status 404 returned error can't find the container with id b926990f47a68be4b8de7f5b618bf47c9176976653dcbccf5c48e49c5b0cd6f1
Sep 18 17:56:39.929340 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:39.929298    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln" event={"ID":"6a06b09b-0510-4c27-8ee0-182b172f05b1","Type":"ContainerStarted","Data":"c3ecad4d8ebb00e145343273db68295daa8a4280d4efac9e95471cd18a422d07"}
Sep 18 17:56:39.930588 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:39.930557    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v" event={"ID":"e451d3c6-52ec-418f-9229-8b9a3588ed3c","Type":"ContainerStarted","Data":"b926990f47a68be4b8de7f5b618bf47c9176976653dcbccf5c48e49c5b0cd6f1"}
Sep 18 17:56:39.953412 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:39.953367    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6b7b7f7d54-5mfln" podStartSLOduration=2.066639767 podStartE2EDuration="4.953351579s" totalImagesPullingTime="2.886711812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:56:39.950788836 +0000 UTC m=+404.028592735" watchObservedRunningTime="2026-09-18 17:56:39.953351579 +0000 UTC m=+404.031155477"
Sep 18 17:56:43.110717 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:43.110673    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 17:56:43.111014 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:43.110755    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 17:56:43.953597 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:43.953563    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v" event={"ID":"e451d3c6-52ec-418f-9229-8b9a3588ed3c","Type":"ContainerStarted","Data":"7ccc1f48ec0bdf61077395c94a21f3d2b60c7cc5af3f3d5e375c5e972b1623ae"}
Sep 18 17:56:43.953807 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:43.953705    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:43.987294 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:43.987239    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v" podStartSLOduration=2.687833337 podStartE2EDuration="5.987221645s" totalImagesPullingTime="3.299388308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:56:43.985022405 +0000 UTC m=+408.062826303" watchObservedRunningTime="2026-09-18 17:56:43.987221645 +0000 UTC m=+408.065025544"
Sep 18 17:56:44.959690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:44.959661    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:56:50.944694 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:50.944634    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5ff77d5444-vgjtt" podUID="6ecacb4a-b719-4db2-8d24-7a41a88e5058" containerName="console" containerID="cri-o://21ac068b1babf74cbd587d6bd1479a7afc416aecedb640d96aa70d25136dea90" gracePeriod=15
Sep 18 17:56:51.211034 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.210958    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:56:51.383233 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.383200    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-service-ca\") pod \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") "
Sep 18 17:56:51.383382 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.383251    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-trusted-ca-bundle\") pod \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") "
Sep 18 17:56:51.383382 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.383275    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-config\") pod \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") "
Sep 18 17:56:51.383382 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.383304    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-oauth-serving-cert\") pod \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") "
Sep 18 17:56:51.383382 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.383325    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wclf2\" (UniqueName: \"kubernetes.io/projected/6ecacb4a-b719-4db2-8d24-7a41a88e5058-kube-api-access-wclf2\") pod \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") "
Sep 18 17:56:51.383382 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.383365    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-serving-cert\") pod \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") "
Sep 18 17:56:51.383628 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.383393    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-oauth-config\") pod \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\" (UID: \"6ecacb4a-b719-4db2-8d24-7a41a88e5058\") "
Sep 18 17:56:51.383786 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.383755    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-service-ca" (OuterVolumeSpecName: "service-ca") pod "6ecacb4a-b719-4db2-8d24-7a41a88e5058" (UID: "6ecacb4a-b719-4db2-8d24-7a41a88e5058"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:56:51.383923 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.383762    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-config" (OuterVolumeSpecName: "console-config") pod "6ecacb4a-b719-4db2-8d24-7a41a88e5058" (UID: "6ecacb4a-b719-4db2-8d24-7a41a88e5058"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:56:51.383923 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.383851    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6ecacb4a-b719-4db2-8d24-7a41a88e5058" (UID: "6ecacb4a-b719-4db2-8d24-7a41a88e5058"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:56:51.383923 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.383879    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6ecacb4a-b719-4db2-8d24-7a41a88e5058" (UID: "6ecacb4a-b719-4db2-8d24-7a41a88e5058"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:56:51.385596 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.385570    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6ecacb4a-b719-4db2-8d24-7a41a88e5058" (UID: "6ecacb4a-b719-4db2-8d24-7a41a88e5058"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:56:51.385690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.385643    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6ecacb4a-b719-4db2-8d24-7a41a88e5058" (UID: "6ecacb4a-b719-4db2-8d24-7a41a88e5058"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:56:51.385690 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.385663    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ecacb4a-b719-4db2-8d24-7a41a88e5058-kube-api-access-wclf2" (OuterVolumeSpecName: "kube-api-access-wclf2") pod "6ecacb4a-b719-4db2-8d24-7a41a88e5058" (UID: "6ecacb4a-b719-4db2-8d24-7a41a88e5058"). InnerVolumeSpecName "kube-api-access-wclf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 17:56:51.484044 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.483967    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wclf2\" (UniqueName: \"kubernetes.io/projected/6ecacb4a-b719-4db2-8d24-7a41a88e5058-kube-api-access-wclf2\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:56:51.484044 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.483994    2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:56:51.484044 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.484005    2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-oauth-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:56:51.484044 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.484015    2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:56:51.484044 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.484024    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-trusted-ca-bundle\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:56:51.484044 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.484033    2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-console-config\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:56:51.484370 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.484052    2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ecacb4a-b719-4db2-8d24-7a41a88e5058-oauth-serving-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:56:51.992701 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.992661    2568 generic.go:356] "Generic (PLEG): container finished" podID="6ecacb4a-b719-4db2-8d24-7a41a88e5058" containerID="21ac068b1babf74cbd587d6bd1479a7afc416aecedb640d96aa70d25136dea90" exitCode=0
Sep 18 17:56:51.993101 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.992706    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5ff77d5444-vgjtt"
Sep 18 17:56:51.993101 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.992751    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ff77d5444-vgjtt" event={"ID":"6ecacb4a-b719-4db2-8d24-7a41a88e5058","Type":"ContainerDied","Data":"21ac068b1babf74cbd587d6bd1479a7afc416aecedb640d96aa70d25136dea90"}
Sep 18 17:56:51.993101 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.992803    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5ff77d5444-vgjtt" event={"ID":"6ecacb4a-b719-4db2-8d24-7a41a88e5058","Type":"ContainerDied","Data":"5ba2bd319077133c7f97562fd524b45264af54c195d29cd62a71465c27b8ecfd"}
Sep 18 17:56:51.993101 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:51.992824    2568 scope.go:117] "RemoveContainer" containerID="21ac068b1babf74cbd587d6bd1479a7afc416aecedb640d96aa70d25136dea90"
Sep 18 17:56:52.001653 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:52.001636    2568 scope.go:117] "RemoveContainer" containerID="21ac068b1babf74cbd587d6bd1479a7afc416aecedb640d96aa70d25136dea90"
Sep 18 17:56:52.001936 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:56:52.001916    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21ac068b1babf74cbd587d6bd1479a7afc416aecedb640d96aa70d25136dea90\": container with ID starting with 21ac068b1babf74cbd587d6bd1479a7afc416aecedb640d96aa70d25136dea90 not found: ID does not exist" containerID="21ac068b1babf74cbd587d6bd1479a7afc416aecedb640d96aa70d25136dea90"
Sep 18 17:56:52.001990 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:52.001946    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21ac068b1babf74cbd587d6bd1479a7afc416aecedb640d96aa70d25136dea90"} err="failed to get container status \"21ac068b1babf74cbd587d6bd1479a7afc416aecedb640d96aa70d25136dea90\": rpc error: code = NotFound desc = could not find container \"21ac068b1babf74cbd587d6bd1479a7afc416aecedb640d96aa70d25136dea90\": container with ID starting with 21ac068b1babf74cbd587d6bd1479a7afc416aecedb640d96aa70d25136dea90 not found: ID does not exist"
Sep 18 17:56:52.030803 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:52.030772    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5ff77d5444-vgjtt"]
Sep 18 17:56:52.044157 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:52.044126    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5ff77d5444-vgjtt"]
Sep 18 17:56:54.044766 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:54.044732    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ecacb4a-b719-4db2-8d24-7a41a88e5058" path="/var/lib/kubelet/pods/6ecacb4a-b719-4db2-8d24-7a41a88e5058/volumes"
Sep 18 17:56:59.648832 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.648796    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr"]
Sep 18 17:56:59.650031 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.650002    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ecacb4a-b719-4db2-8d24-7a41a88e5058" containerName="console"
Sep 18 17:56:59.650202 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.650168    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ecacb4a-b719-4db2-8d24-7a41a88e5058" containerName="console"
Sep 18 17:56:59.650466 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.650451    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ecacb4a-b719-4db2-8d24-7a41a88e5058" containerName="console"
Sep 18 17:56:59.655702 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.655682    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz"]
Sep 18 17:56:59.655819 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.655803    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr"
Sep 18 17:56:59.658369 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.658349    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz"
Sep 18 17:56:59.661811 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.661791    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 18 17:56:59.662083 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.662065    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-h72j7\""
Sep 18 17:56:59.662904 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.662885    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 18 17:56:59.664930 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.664911    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4bfb2\""
Sep 18 17:56:59.680608 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.680583    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr"]
Sep 18 17:56:59.682933 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.682909    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz"]
Sep 18 17:56:59.752684 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.752653    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bdfec0cd-ab90-4bea-b28a-b4adbdc15a16-cert\") pod \"kserve-controller-manager-d6f4b65b8-w9zjr\" (UID: \"bdfec0cd-ab90-4bea-b28a-b4adbdc15a16\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr"
Sep 18 17:56:59.752843 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.752704    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jtng\" (UniqueName: \"kubernetes.io/projected/36258bba-ac53-45cd-b05c-248d4f74caf7-kube-api-access-4jtng\") pod \"llmisvc-controller-manager-777c6bd7d5-pnlsz\" (UID: \"36258bba-ac53-45cd-b05c-248d4f74caf7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz"
Sep 18 17:56:59.752843 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.752728    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsmd2\" (UniqueName: \"kubernetes.io/projected/bdfec0cd-ab90-4bea-b28a-b4adbdc15a16-kube-api-access-bsmd2\") pod \"kserve-controller-manager-d6f4b65b8-w9zjr\" (UID: \"bdfec0cd-ab90-4bea-b28a-b4adbdc15a16\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr"
Sep 18 17:56:59.752843 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.752818    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36258bba-ac53-45cd-b05c-248d4f74caf7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pnlsz\" (UID: \"36258bba-ac53-45cd-b05c-248d4f74caf7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz"
Sep 18 17:56:59.766049 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.766018    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"]
Sep 18 17:56:59.766269 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.766246    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v" podUID="e451d3c6-52ec-418f-9229-8b9a3588ed3c" containerName="discovery" containerID="cri-o://7ccc1f48ec0bdf61077395c94a21f3d2b60c7cc5af3f3d5e375c5e972b1623ae" gracePeriod=30
Sep 18 17:56:59.853497 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.853466    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jtng\" (UniqueName: \"kubernetes.io/projected/36258bba-ac53-45cd-b05c-248d4f74caf7-kube-api-access-4jtng\") pod \"llmisvc-controller-manager-777c6bd7d5-pnlsz\" (UID: \"36258bba-ac53-45cd-b05c-248d4f74caf7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz"
Sep 18 17:56:59.853653 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.853505    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsmd2\" (UniqueName: \"kubernetes.io/projected/bdfec0cd-ab90-4bea-b28a-b4adbdc15a16-kube-api-access-bsmd2\") pod \"kserve-controller-manager-d6f4b65b8-w9zjr\" (UID: \"bdfec0cd-ab90-4bea-b28a-b4adbdc15a16\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr"
Sep 18 17:56:59.853653 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.853541    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36258bba-ac53-45cd-b05c-248d4f74caf7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pnlsz\" (UID: \"36258bba-ac53-45cd-b05c-248d4f74caf7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz"
Sep 18 17:56:59.853653 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.853575    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bdfec0cd-ab90-4bea-b28a-b4adbdc15a16-cert\") pod \"kserve-controller-manager-d6f4b65b8-w9zjr\" (UID: \"bdfec0cd-ab90-4bea-b28a-b4adbdc15a16\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr"
Sep 18 17:56:59.856065 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.856039    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36258bba-ac53-45cd-b05c-248d4f74caf7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pnlsz\" (UID: \"36258bba-ac53-45cd-b05c-248d4f74caf7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz"
Sep 18 17:56:59.856191 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.856068    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bdfec0cd-ab90-4bea-b28a-b4adbdc15a16-cert\") pod \"kserve-controller-manager-d6f4b65b8-w9zjr\" (UID: \"bdfec0cd-ab90-4bea-b28a-b4adbdc15a16\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr"
Sep 18 17:56:59.869560 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.869537    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jtng\" (UniqueName: \"kubernetes.io/projected/36258bba-ac53-45cd-b05c-248d4f74caf7-kube-api-access-4jtng\") pod \"llmisvc-controller-manager-777c6bd7d5-pnlsz\" (UID: \"36258bba-ac53-45cd-b05c-248d4f74caf7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz"
Sep 18 17:56:59.875971 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.875943    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsmd2\" (UniqueName: \"kubernetes.io/projected/bdfec0cd-ab90-4bea-b28a-b4adbdc15a16-kube-api-access-bsmd2\") pod \"kserve-controller-manager-d6f4b65b8-w9zjr\" (UID: \"bdfec0cd-ab90-4bea-b28a-b4adbdc15a16\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr"
Sep 18 17:56:59.967943 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.967905    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr"
Sep 18 17:56:59.973501 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:56:59.973473    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz"
Sep 18 17:57:00.008907 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.008853    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:57:00.035793 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.035442    2568 generic.go:356] "Generic (PLEG): container finished" podID="e451d3c6-52ec-418f-9229-8b9a3588ed3c" containerID="7ccc1f48ec0bdf61077395c94a21f3d2b60c7cc5af3f3d5e375c5e972b1623ae" exitCode=0
Sep 18 17:57:00.035793 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.035569    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v" event={"ID":"e451d3c6-52ec-418f-9229-8b9a3588ed3c","Type":"ContainerDied","Data":"7ccc1f48ec0bdf61077395c94a21f3d2b60c7cc5af3f3d5e375c5e972b1623ae"}
Sep 18 17:57:00.035793 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.035617    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v" event={"ID":"e451d3c6-52ec-418f-9229-8b9a3588ed3c","Type":"ContainerDied","Data":"b926990f47a68be4b8de7f5b618bf47c9176976653dcbccf5c48e49c5b0cd6f1"}
Sep 18 17:57:00.035793 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.035646    2568 scope.go:117] "RemoveContainer" containerID="7ccc1f48ec0bdf61077395c94a21f3d2b60c7cc5af3f3d5e375c5e972b1623ae"
Sep 18 17:57:00.036528 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.036373    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"
Sep 18 17:57:00.047496 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.047431    2568 scope.go:117] "RemoveContainer" containerID="7ccc1f48ec0bdf61077395c94a21f3d2b60c7cc5af3f3d5e375c5e972b1623ae"
Sep 18 17:57:00.047722 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:57:00.047701    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ccc1f48ec0bdf61077395c94a21f3d2b60c7cc5af3f3d5e375c5e972b1623ae\": container with ID starting with 7ccc1f48ec0bdf61077395c94a21f3d2b60c7cc5af3f3d5e375c5e972b1623ae not found: ID does not exist" containerID="7ccc1f48ec0bdf61077395c94a21f3d2b60c7cc5af3f3d5e375c5e972b1623ae"
Sep 18 17:57:00.047797 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.047734    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ccc1f48ec0bdf61077395c94a21f3d2b60c7cc5af3f3d5e375c5e972b1623ae"} err="failed to get container status \"7ccc1f48ec0bdf61077395c94a21f3d2b60c7cc5af3f3d5e375c5e972b1623ae\": rpc error: code = NotFound desc = could not find container \"7ccc1f48ec0bdf61077395c94a21f3d2b60c7cc5af3f3d5e375c5e972b1623ae\": container with ID starting with 7ccc1f48ec0bdf61077395c94a21f3d2b60c7cc5af3f3d5e375c5e972b1623ae not found: ID does not exist"
Sep 18 17:57:00.055596 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.055572    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-kubeconfig\") pod \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") "
Sep 18 17:57:00.055673 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.055629    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-cacerts\") pod \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") "
Sep 18 17:57:00.055734 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.055677    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e451d3c6-52ec-418f-9229-8b9a3588ed3c-local-certs\") pod \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") "
Sep 18 17:57:00.055734 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.055702    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-csr-dns-cert\") pod \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") "
Sep 18 17:57:00.055840 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.055735    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-csr-ca-configmap\") pod \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") "
Sep 18 17:57:00.055840 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.055778    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-token\") pod \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") "
Sep 18 17:57:00.055938 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.055861    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxbxg\" (UniqueName: \"kubernetes.io/projected/e451d3c6-52ec-418f-9229-8b9a3588ed3c-kube-api-access-zxbxg\") pod \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\" (UID: \"e451d3c6-52ec-418f-9229-8b9a3588ed3c\") "
Sep 18 17:57:00.056287 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.056221    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e451d3c6-52ec-418f-9229-8b9a3588ed3c" (UID: "e451d3c6-52ec-418f-9229-8b9a3588ed3c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 17:57:00.059774 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.059742    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e451d3c6-52ec-418f-9229-8b9a3588ed3c" (UID: "e451d3c6-52ec-418f-9229-8b9a3588ed3c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:57:00.059885 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.059865    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e451d3c6-52ec-418f-9229-8b9a3588ed3c" (UID: "e451d3c6-52ec-418f-9229-8b9a3588ed3c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:57:00.059948 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.059881    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-cacerts" (OuterVolumeSpecName: "cacerts") pod "e451d3c6-52ec-418f-9229-8b9a3588ed3c" (UID: "e451d3c6-52ec-418f-9229-8b9a3588ed3c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 17:57:00.060006 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.059950    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-token" (OuterVolumeSpecName: "istio-token") pod "e451d3c6-52ec-418f-9229-8b9a3588ed3c" (UID: "e451d3c6-52ec-418f-9229-8b9a3588ed3c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 17:57:00.060006 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.059970    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e451d3c6-52ec-418f-9229-8b9a3588ed3c-local-certs" (OuterVolumeSpecName: "local-certs") pod "e451d3c6-52ec-418f-9229-8b9a3588ed3c" (UID: "e451d3c6-52ec-418f-9229-8b9a3588ed3c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 17:57:00.060220 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.060202    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e451d3c6-52ec-418f-9229-8b9a3588ed3c-kube-api-access-zxbxg" (OuterVolumeSpecName: "kube-api-access-zxbxg") pod "e451d3c6-52ec-418f-9229-8b9a3588ed3c" (UID: "e451d3c6-52ec-418f-9229-8b9a3588ed3c"). InnerVolumeSpecName "kube-api-access-zxbxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 17:57:00.156919 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.156888    2568 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-kubeconfig\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:57:00.156919 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.156913    2568 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-cacerts\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:57:00.156919 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.156921    2568 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e451d3c6-52ec-418f-9229-8b9a3588ed3c-local-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:57:00.157135 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.156930    2568 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-csr-dns-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:57:00.157135 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.156940    2568 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-csr-ca-configmap\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:57:00.157135 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.156950    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e451d3c6-52ec-418f-9229-8b9a3588ed3c-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:57:00.157135 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.156959    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxbxg\" (UniqueName: \"kubernetes.io/projected/e451d3c6-52ec-418f-9229-8b9a3588ed3c-kube-api-access-zxbxg\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:57:00.275426 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.275400    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz"]
Sep 18 17:57:00.276889 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:57:00.276864    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod36258bba_ac53_45cd_b05c_248d4f74caf7.slice/crio-762a4bc62fcea254aae410d4bd17e9bf6fe153ae7e72bd22dd21aa34f3624031 WatchSource:0}: Error finding container 762a4bc62fcea254aae410d4bd17e9bf6fe153ae7e72bd22dd21aa34f3624031: Status 404 returned error can't find the container with id 762a4bc62fcea254aae410d4bd17e9bf6fe153ae7e72bd22dd21aa34f3624031
Sep 18 17:57:00.293279 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.293255    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr"]
Sep 18 17:57:00.296047 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:57:00.296021    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdfec0cd_ab90_4bea_b28a_b4adbdc15a16.slice/crio-8612918db5ed80aaf590f385a42a7994a8755faa88cbdbbbcdad2e137c0bb9ea WatchSource:0}: Error finding container 8612918db5ed80aaf590f385a42a7994a8755faa88cbdbbbcdad2e137c0bb9ea: Status 404 returned error can't find the container with id 8612918db5ed80aaf590f385a42a7994a8755faa88cbdbbbcdad2e137c0bb9ea
Sep 18 17:57:00.383937 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.383897    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"]
Sep 18 17:57:00.402093 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:00.402063    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v"]
Sep 18 17:57:01.040957 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:01.040928    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr" event={"ID":"bdfec0cd-ab90-4bea-b28a-b4adbdc15a16","Type":"ContainerStarted","Data":"8612918db5ed80aaf590f385a42a7994a8755faa88cbdbbbcdad2e137c0bb9ea"}
Sep 18 17:57:01.046095 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:01.046068    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz" event={"ID":"36258bba-ac53-45cd-b05c-248d4f74caf7","Type":"ContainerStarted","Data":"762a4bc62fcea254aae410d4bd17e9bf6fe153ae7e72bd22dd21aa34f3624031"}
Sep 18 17:57:02.044472 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:02.044435    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e451d3c6-52ec-418f-9229-8b9a3588ed3c" path="/var/lib/kubelet/pods/e451d3c6-52ec-418f-9229-8b9a3588ed3c/volumes"
Sep 18 17:57:04.061501 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:04.061452    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz" event={"ID":"36258bba-ac53-45cd-b05c-248d4f74caf7","Type":"ContainerStarted","Data":"f104cfda22d082f8e13853c885ea227438b04f9033cfdc65b87b1a50039c3768"}
Sep 18 17:57:04.062101 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:04.061563    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz"
Sep 18 17:57:04.063279 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:04.063254    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr" event={"ID":"bdfec0cd-ab90-4bea-b28a-b4adbdc15a16","Type":"ContainerStarted","Data":"dc02d948bd33f27b898200a9a6fe06a6961a97f9ca3505d865d5d4ce072b35d5"}
Sep 18 17:57:04.063425 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:04.063385    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr"
Sep 18 17:57:04.258485 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:04.258431    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr" podStartSLOduration=2.310360642 podStartE2EDuration="5.25841374s" totalImagesPullingTime="2.948053098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:56:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:57:04.258229411 +0000 UTC m=+428.336033312" watchObservedRunningTime="2026-09-18 17:57:04.25841374 +0000 UTC m=+428.336217640"
Sep 18 17:57:04.259140 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:04.259112    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz" podStartSLOduration=2.246913198 podStartE2EDuration="5.259102051s" totalImagesPullingTime="3.012188853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:56:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:57:04.139195097 +0000 UTC m=+428.216998988" watchObservedRunningTime="2026-09-18 17:57:04.259102051 +0000 UTC m=+428.336905979"
Sep 18 17:57:04.959797 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:04.959748    2568 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-5m68v container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.29:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 17:57:04.959990 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:04.959831    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-5m68v" podUID="e451d3c6-52ec-418f-9229-8b9a3588ed3c" containerName="discovery" probeResult="failure" output="Get \"http://10.134.0.29:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 17:57:35.070379 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:35.070350    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnlsz"
Sep 18 17:57:35.073741 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:57:35.073718    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9zjr"
Sep 18 17:58:08.642034 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.641995    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"]
Sep 18 17:58:08.642569 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.642373    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e451d3c6-52ec-418f-9229-8b9a3588ed3c" containerName="discovery"
Sep 18 17:58:08.642569 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.642384    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e451d3c6-52ec-418f-9229-8b9a3588ed3c" containerName="discovery"
Sep 18 17:58:08.642569 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.642436    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e451d3c6-52ec-418f-9229-8b9a3588ed3c" containerName="discovery"
Sep 18 17:58:08.646544 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.646523    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:08.650609 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.650586    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 18 17:58:08.650609 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.650599    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 18 17:58:08.650803 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.650667    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9x8zw\""
Sep 18 17:58:08.685517 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.685488    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"]
Sep 18 17:58:08.759088 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.759049    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c61598-9e16-46bb-929d-9f2bb9879549-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-h4jrg\" (UID: \"34c61598-9e16-46bb-929d-9f2bb9879549\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:08.759088 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.759089    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86bpb\" (UniqueName: \"kubernetes.io/projected/34c61598-9e16-46bb-929d-9f2bb9879549-kube-api-access-86bpb\") pod \"kuadrant-operator-controller-manager-76f8f686bf-h4jrg\" (UID: \"34c61598-9e16-46bb-929d-9f2bb9879549\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:08.860364 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.860323    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c61598-9e16-46bb-929d-9f2bb9879549-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-h4jrg\" (UID: \"34c61598-9e16-46bb-929d-9f2bb9879549\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:08.860364 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.860365    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86bpb\" (UniqueName: \"kubernetes.io/projected/34c61598-9e16-46bb-929d-9f2bb9879549-kube-api-access-86bpb\") pod \"kuadrant-operator-controller-manager-76f8f686bf-h4jrg\" (UID: \"34c61598-9e16-46bb-929d-9f2bb9879549\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:08.860737 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.860718    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c61598-9e16-46bb-929d-9f2bb9879549-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-h4jrg\" (UID: \"34c61598-9e16-46bb-929d-9f2bb9879549\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:08.872824 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.872789    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86bpb\" (UniqueName: \"kubernetes.io/projected/34c61598-9e16-46bb-929d-9f2bb9879549-kube-api-access-86bpb\") pod \"kuadrant-operator-controller-manager-76f8f686bf-h4jrg\" (UID: \"34c61598-9e16-46bb-929d-9f2bb9879549\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:08.957081 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:08.957044    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:09.266407 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:09.266331    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"]
Sep 18 17:58:09.269470 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:58:09.269440    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34c61598_9e16_46bb_929d_9f2bb9879549.slice/crio-f0dd7b88a5184a1d159639bb18cf8545506b461da1ef8aec0c33d91347f6a4ed WatchSource:0}: Error finding container f0dd7b88a5184a1d159639bb18cf8545506b461da1ef8aec0c33d91347f6a4ed: Status 404 returned error can't find the container with id f0dd7b88a5184a1d159639bb18cf8545506b461da1ef8aec0c33d91347f6a4ed
Sep 18 17:58:09.350464 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:09.350422    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg" event={"ID":"34c61598-9e16-46bb-929d-9f2bb9879549","Type":"ContainerStarted","Data":"f0dd7b88a5184a1d159639bb18cf8545506b461da1ef8aec0c33d91347f6a4ed"}
Sep 18 17:58:10.653969 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:10.653927    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kgrb2"]
Sep 18 17:58:10.657506 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:10.657478    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kgrb2"
Sep 18 17:58:10.660471 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:10.660448    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-l628g\""
Sep 18 17:58:10.662895 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:10.662869    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 18 17:58:10.676936 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:10.676866    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kgrb2"]
Sep 18 17:58:10.778119 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:10.778074    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj2mv\" (UniqueName: \"kubernetes.io/projected/77d85aaf-ca7c-4590-b343-4aec7e982f47-kube-api-access-lj2mv\") pod \"dns-operator-controller-manager-7b84cd7c9c-kgrb2\" (UID: \"77d85aaf-ca7c-4590-b343-4aec7e982f47\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kgrb2"
Sep 18 17:58:10.879525 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:10.879472    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj2mv\" (UniqueName: \"kubernetes.io/projected/77d85aaf-ca7c-4590-b343-4aec7e982f47-kube-api-access-lj2mv\") pod \"dns-operator-controller-manager-7b84cd7c9c-kgrb2\" (UID: \"77d85aaf-ca7c-4590-b343-4aec7e982f47\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kgrb2"
Sep 18 17:58:10.892504 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:10.892466    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj2mv\" (UniqueName: \"kubernetes.io/projected/77d85aaf-ca7c-4590-b343-4aec7e982f47-kube-api-access-lj2mv\") pod \"dns-operator-controller-manager-7b84cd7c9c-kgrb2\" (UID: \"77d85aaf-ca7c-4590-b343-4aec7e982f47\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kgrb2"
Sep 18 17:58:10.970647 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:10.970523    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kgrb2"
Sep 18 17:58:11.264461 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:11.264431    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kgrb2"]
Sep 18 17:58:11.806299 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:58:11.806257    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d85aaf_ca7c_4590_b343_4aec7e982f47.slice/crio-495d467da62a0736f9bdac784a40ceb192b2fe96d55288a165b823931381306f WatchSource:0}: Error finding container 495d467da62a0736f9bdac784a40ceb192b2fe96d55288a165b823931381306f: Status 404 returned error can't find the container with id 495d467da62a0736f9bdac784a40ceb192b2fe96d55288a165b823931381306f
Sep 18 17:58:12.372240 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:12.372204    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kgrb2" event={"ID":"77d85aaf-ca7c-4590-b343-4aec7e982f47","Type":"ContainerStarted","Data":"495d467da62a0736f9bdac784a40ceb192b2fe96d55288a165b823931381306f"}
Sep 18 17:58:14.028358 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:14.028322    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-7mkn8"]
Sep 18 17:58:14.032073 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:14.032040    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-7mkn8"
Sep 18 17:58:14.035150 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:14.035129    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-j7l7m\""
Sep 18 17:58:14.047111 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:14.047084    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-7mkn8"]
Sep 18 17:58:14.106709 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:14.106673    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99nbf\" (UniqueName: \"kubernetes.io/projected/f7e509c0-a896-4ebf-89f5-e510eb9f8796-kube-api-access-99nbf\") pod \"authorino-operator-76b478bd78-7mkn8\" (UID: \"f7e509c0-a896-4ebf-89f5-e510eb9f8796\") " pod="kuadrant-system/authorino-operator-76b478bd78-7mkn8"
Sep 18 17:58:14.207419 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:14.207377    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99nbf\" (UniqueName: \"kubernetes.io/projected/f7e509c0-a896-4ebf-89f5-e510eb9f8796-kube-api-access-99nbf\") pod \"authorino-operator-76b478bd78-7mkn8\" (UID: \"f7e509c0-a896-4ebf-89f5-e510eb9f8796\") " pod="kuadrant-system/authorino-operator-76b478bd78-7mkn8"
Sep 18 17:58:14.218396 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:14.218363    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99nbf\" (UniqueName: \"kubernetes.io/projected/f7e509c0-a896-4ebf-89f5-e510eb9f8796-kube-api-access-99nbf\") pod \"authorino-operator-76b478bd78-7mkn8\" (UID: \"f7e509c0-a896-4ebf-89f5-e510eb9f8796\") " pod="kuadrant-system/authorino-operator-76b478bd78-7mkn8"
Sep 18 17:58:14.344589 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:14.344497    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-7mkn8"
Sep 18 17:58:14.387572 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:14.386997    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg" event={"ID":"34c61598-9e16-46bb-929d-9f2bb9879549","Type":"ContainerStarted","Data":"eee02d397ee91b9e667799f1ad87b3ac8cbcafcf5e5018e5ffdaab3e2af77c91"}
Sep 18 17:58:14.387572 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:14.387148    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:14.415354 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:14.415277    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg" podStartSLOduration=2.196110686 podStartE2EDuration="6.415252052s" totalImagesPullingTime="4.219141366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:58:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:58:14.412924991 +0000 UTC m=+498.490728903" watchObservedRunningTime="2026-09-18 17:58:14.415252052 +0000 UTC m=+498.493055952"
Sep 18 17:58:14.642445 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:14.642361    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-7mkn8"]
Sep 18 17:58:14.774661 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:58:14.774624    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e509c0_a896_4ebf_89f5_e510eb9f8796.slice/crio-d17e2e134fb84267f9312b1bf683195e6346a8beae636f1f4059aca4d95918ef WatchSource:0}: Error finding container d17e2e134fb84267f9312b1bf683195e6346a8beae636f1f4059aca4d95918ef: Status 404 returned error can't find the container with id d17e2e134fb84267f9312b1bf683195e6346a8beae636f1f4059aca4d95918ef
Sep 18 17:58:15.393087 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:15.393039    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-7mkn8" event={"ID":"f7e509c0-a896-4ebf-89f5-e510eb9f8796","Type":"ContainerStarted","Data":"d17e2e134fb84267f9312b1bf683195e6346a8beae636f1f4059aca4d95918ef"}
Sep 18 17:58:15.394834 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:15.394808    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kgrb2" event={"ID":"77d85aaf-ca7c-4590-b343-4aec7e982f47","Type":"ContainerStarted","Data":"80c190260bb6ce5afe34e1a6967d3bf12a9100244ba3b9ec6798f49ebc238fd2"}
Sep 18 17:58:15.395089 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:15.395071    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kgrb2"
Sep 18 17:58:15.438968 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:15.438908    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kgrb2" podStartSLOduration=2.4163727870000002 podStartE2EDuration="5.438889289s" totalImagesPullingTime="3.022516502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:58:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:58:15.43862303 +0000 UTC m=+499.516426927" watchObservedRunningTime="2026-09-18 17:58:15.438889289 +0000 UTC m=+499.516693178"
Sep 18 17:58:16.175387 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.175347    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"]
Sep 18 17:58:16.179700 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.179674    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:16.183305 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.183273    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-xfvl4\""
Sep 18 17:58:16.203510 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.203477    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"]
Sep 18 17:58:16.328518 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.328479    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmzks\" (UniqueName: \"kubernetes.io/projected/e2d8c40c-18cb-4118-af3b-c21c746afea7-kube-api-access-tmzks\") pod \"limitador-operator-controller-manager-74495c69f8-kj66h\" (UID: \"e2d8c40c-18cb-4118-af3b-c21c746afea7\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:16.429996 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.429966    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmzks\" (UniqueName: \"kubernetes.io/projected/e2d8c40c-18cb-4118-af3b-c21c746afea7-kube-api-access-tmzks\") pod \"limitador-operator-controller-manager-74495c69f8-kj66h\" (UID: \"e2d8c40c-18cb-4118-af3b-c21c746afea7\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:16.439268 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.439247    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmzks\" (UniqueName: \"kubernetes.io/projected/e2d8c40c-18cb-4118-af3b-c21c746afea7-kube-api-access-tmzks\") pod \"limitador-operator-controller-manager-74495c69f8-kj66h\" (UID: \"e2d8c40c-18cb-4118-af3b-c21c746afea7\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:16.492746 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.492691    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:16.676236 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.676196    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9"]
Sep 18 17:58:16.681039 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.680973    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9"
Sep 18 17:58:16.684153 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.684123    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 18 17:58:16.684315 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.684262    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 18 17:58:16.684570 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.684531    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-v8xtm\""
Sep 18 17:58:16.696591 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.696561    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9"]
Sep 18 17:58:16.795188 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.795154    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"]
Sep 18 17:58:16.800820 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:58:16.800791    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d8c40c_18cb_4118_af3b_c21c746afea7.slice/crio-2b6e9a687524cbb36066473a4dd199584e485dcb9f88cd003db0f8a1d1773f92 WatchSource:0}: Error finding container 2b6e9a687524cbb36066473a4dd199584e485dcb9f88cd003db0f8a1d1773f92: Status 404 returned error can't find the container with id 2b6e9a687524cbb36066473a4dd199584e485dcb9f88cd003db0f8a1d1773f92
Sep 18 17:58:16.834948 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.834907    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhdsn\" (UniqueName: \"kubernetes.io/projected/a95f0f33-d61f-449f-9d38-a103d5ec4709-kube-api-access-lhdsn\") pod \"kuadrant-console-plugin-fbdd4f445-5r8g9\" (UID: \"a95f0f33-d61f-449f-9d38-a103d5ec4709\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9"
Sep 18 17:58:16.834948 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.834956    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a95f0f33-d61f-449f-9d38-a103d5ec4709-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-5r8g9\" (UID: \"a95f0f33-d61f-449f-9d38-a103d5ec4709\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9"
Sep 18 17:58:16.835201 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.834989    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a95f0f33-d61f-449f-9d38-a103d5ec4709-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-5r8g9\" (UID: \"a95f0f33-d61f-449f-9d38-a103d5ec4709\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9"
Sep 18 17:58:16.936479 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.936389    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhdsn\" (UniqueName: \"kubernetes.io/projected/a95f0f33-d61f-449f-9d38-a103d5ec4709-kube-api-access-lhdsn\") pod \"kuadrant-console-plugin-fbdd4f445-5r8g9\" (UID: \"a95f0f33-d61f-449f-9d38-a103d5ec4709\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9"
Sep 18 17:58:16.936479 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.936430    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a95f0f33-d61f-449f-9d38-a103d5ec4709-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-5r8g9\" (UID: \"a95f0f33-d61f-449f-9d38-a103d5ec4709\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9"
Sep 18 17:58:16.936676 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:16.936530    2568 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 18 17:58:16.936676 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:16.936594    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a95f0f33-d61f-449f-9d38-a103d5ec4709-plugin-serving-cert podName:a95f0f33-d61f-449f-9d38-a103d5ec4709 nodeName:}" failed. No retries permitted until 2026-09-18 17:58:17.436573379 +0000 UTC m=+501.514377272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/a95f0f33-d61f-449f-9d38-a103d5ec4709-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-5r8g9" (UID: "a95f0f33-d61f-449f-9d38-a103d5ec4709") : secret "plugin-serving-cert" not found
Sep 18 17:58:16.936756 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.936688    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a95f0f33-d61f-449f-9d38-a103d5ec4709-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-5r8g9\" (UID: \"a95f0f33-d61f-449f-9d38-a103d5ec4709\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9"
Sep 18 17:58:16.937341 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.937321    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a95f0f33-d61f-449f-9d38-a103d5ec4709-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-5r8g9\" (UID: \"a95f0f33-d61f-449f-9d38-a103d5ec4709\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9"
Sep 18 17:58:16.960677 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:16.960650    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhdsn\" (UniqueName: \"kubernetes.io/projected/a95f0f33-d61f-449f-9d38-a103d5ec4709-kube-api-access-lhdsn\") pod \"kuadrant-console-plugin-fbdd4f445-5r8g9\" (UID: \"a95f0f33-d61f-449f-9d38-a103d5ec4709\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9"
Sep 18 17:58:17.407929 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:17.407884    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-7mkn8" event={"ID":"f7e509c0-a896-4ebf-89f5-e510eb9f8796","Type":"ContainerStarted","Data":"90db6f486b44666e041b3cd23b45223ef256e2a4e31369e598fbfd1bf630c189"}
Sep 18 17:58:17.408089 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:17.408033    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-7mkn8"
Sep 18 17:58:17.409310 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:17.409289    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h" event={"ID":"e2d8c40c-18cb-4118-af3b-c21c746afea7","Type":"ContainerStarted","Data":"2b6e9a687524cbb36066473a4dd199584e485dcb9f88cd003db0f8a1d1773f92"}
Sep 18 17:58:17.440277 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:17.440243    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a95f0f33-d61f-449f-9d38-a103d5ec4709-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-5r8g9\" (UID: \"a95f0f33-d61f-449f-9d38-a103d5ec4709\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9"
Sep 18 17:58:17.442660 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:17.442638    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a95f0f33-d61f-449f-9d38-a103d5ec4709-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-5r8g9\" (UID: \"a95f0f33-d61f-449f-9d38-a103d5ec4709\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9"
Sep 18 17:58:17.455692 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:17.455642    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-7mkn8" podStartSLOduration=2.8527153419999998 podStartE2EDuration="4.455629061s" totalImagesPullingTime="1.602913719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:58:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:58:17.454601723 +0000 UTC m=+501.532405621" watchObservedRunningTime="2026-09-18 17:58:17.455629061 +0000 UTC m=+501.533432960"
Sep 18 17:58:17.590740 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:17.590704    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9"
Sep 18 17:58:17.867425 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:17.867275    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9"]
Sep 18 17:58:17.872284 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:58:17.872254    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda95f0f33_d61f_449f_9d38_a103d5ec4709.slice/crio-460926cb9ab01d79c2777d30d0c88ec5c0be524019b9ca3326b0e16d81fb2abe WatchSource:0}: Error finding container 460926cb9ab01d79c2777d30d0c88ec5c0be524019b9ca3326b0e16d81fb2abe: Status 404 returned error can't find the container with id 460926cb9ab01d79c2777d30d0c88ec5c0be524019b9ca3326b0e16d81fb2abe
Sep 18 17:58:18.417418 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:18.417372    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9" event={"ID":"a95f0f33-d61f-449f-9d38-a103d5ec4709","Type":"ContainerStarted","Data":"460926cb9ab01d79c2777d30d0c88ec5c0be524019b9ca3326b0e16d81fb2abe"}
Sep 18 17:58:19.425225 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:19.425161    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h" event={"ID":"e2d8c40c-18cb-4118-af3b-c21c746afea7","Type":"ContainerStarted","Data":"d2ffa32d636afdc3bbd6e0481e710a961f90419d67ab30a3541840d3fa0846ac"}
Sep 18 17:58:19.425761 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:19.425299    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:19.444680 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:19.444596    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h" podStartSLOduration=1.648204333 podStartE2EDuration="3.444578026s" totalImagesPullingTime="1.796373693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:58:19.442766847 +0000 UTC m=+503.520570746" watchObservedRunningTime="2026-09-18 17:58:19.444578026 +0000 UTC m=+503.522381922"
Sep 18 17:58:23.447259 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:23.447220    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9" event={"ID":"a95f0f33-d61f-449f-9d38-a103d5ec4709","Type":"ContainerStarted","Data":"1d2c8ac4ad465eb307e3d7c582ea88d206e4385ad18684c237fd5c6de549ff67"}
Sep 18 17:58:25.397922 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:25.397889    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:25.420393 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:25.420330    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5r8g9" podStartSLOduration=4.850769267 podStartE2EDuration="9.420310016s" totalImagesPullingTime="4.569540749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:58:23.497975425 +0000 UTC m=+507.575779323" watchObservedRunningTime="2026-09-18 17:58:25.420310016 +0000 UTC m=+509.498113914"
Sep 18 17:58:26.402598 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:26.402558    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-kgrb2"
Sep 18 17:58:27.719840 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.719808    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"]
Sep 18 17:58:27.720296 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.720009    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg" podUID="34c61598-9e16-46bb-929d-9f2bb9879549" containerName="manager" containerID="cri-o://eee02d397ee91b9e667799f1ad87b3ac8cbcafcf5e5018e5ffdaab3e2af77c91" gracePeriod=2
Sep 18 17:58:27.730456 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.730429    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"]
Sep 18 17:58:27.732472 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:27.732431    2568 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"34c61598-9e16-46bb-929d-9f2bb9879549\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-h4jrg\": pods \"kuadrant-operator-controller-manager-76f8f686bf-h4jrg\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:27.746153 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.746125    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"]
Sep 18 17:58:27.746417 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.746368    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h" podUID="e2d8c40c-18cb-4118-af3b-c21c746afea7" containerName="manager" containerID="cri-o://d2ffa32d636afdc3bbd6e0481e710a961f90419d67ab30a3541840d3fa0846ac" gracePeriod=2
Sep 18 17:58:27.748697 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.748657    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:27.762966 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.762930    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"]
Sep 18 17:58:27.772288 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:27.772244    2568 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e2d8c40c-18cb-4118-af3b-c21c746afea7\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-kj66h\": pods \"limitador-operator-controller-manager-74495c69f8-kj66h\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:27.782850 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.782816    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"]
Sep 18 17:58:27.783147 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:27.783117    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-h4jrg\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="34c61598-9e16-46bb-929d-9f2bb9879549" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:27.783377 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.783357    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c61598-9e16-46bb-929d-9f2bb9879549" containerName="manager"
Sep 18 17:58:27.783492 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.783475    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c61598-9e16-46bb-929d-9f2bb9879549" containerName="manager"
Sep 18 17:58:27.783578 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.783515    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2d8c40c-18cb-4118-af3b-c21c746afea7" containerName="manager"
Sep 18 17:58:27.783578 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.783525    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2d8c40c-18cb-4118-af3b-c21c746afea7" containerName="manager"
Sep 18 17:58:27.783677 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.783646    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2d8c40c-18cb-4118-af3b-c21c746afea7" containerName="manager"
Sep 18 17:58:27.783677 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.783661    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="34c61598-9e16-46bb-929d-9f2bb9879549" containerName="manager"
Sep 18 17:58:27.786802 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.786781    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"
Sep 18 17:58:27.788334 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:27.788309    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-kj66h\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="e2d8c40c-18cb-4118-af3b-c21c746afea7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:27.790852 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:27.790820    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-h4jrg\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="34c61598-9e16-46bb-929d-9f2bb9879549" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:27.791425 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.791404    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rl2wg"]
Sep 18 17:58:27.794600 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.794581    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rl2wg"
Sep 18 17:58:27.795669 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:27.795647    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-kj66h\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="e2d8c40c-18cb-4118-af3b-c21c746afea7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:27.806278 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:27.806246    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-h4jrg\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="34c61598-9e16-46bb-929d-9f2bb9879549" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:27.811250 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.811220    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"]
Sep 18 17:58:27.814580 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.814550    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rl2wg"]
Sep 18 17:58:27.866946 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:27.866905    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-kj66h\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="e2d8c40c-18cb-4118-af3b-c21c746afea7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:27.869759 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:27.869722    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-h4jrg\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="34c61598-9e16-46bb-929d-9f2bb9879549" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:27.937707 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.937672    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgcw5\" (UniqueName: \"kubernetes.io/projected/d0cbd02b-c258-41ee-8a7b-490bcc517da9-kube-api-access-wgcw5\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9sdzb\" (UID: \"d0cbd02b-c258-41ee-8a7b-490bcc517da9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"
Sep 18 17:58:27.937707 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.937721    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0cbd02b-c258-41ee-8a7b-490bcc517da9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9sdzb\" (UID: \"d0cbd02b-c258-41ee-8a7b-490bcc517da9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"
Sep 18 17:58:27.937965 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.937826    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b9x8\" (UniqueName: \"kubernetes.io/projected/952b6ec8-f905-4d6c-905f-9c2844352125-kube-api-access-5b9x8\") pod \"limitador-operator-controller-manager-74495c69f8-rl2wg\" (UID: \"952b6ec8-f905-4d6c-905f-9c2844352125\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rl2wg"
Sep 18 17:58:27.992228 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.992202    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:27.994552 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:27.994523    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-kj66h\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="e2d8c40c-18cb-4118-af3b-c21c746afea7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:27.994848 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:27.994830    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:27.996151 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:27.996130    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-h4jrg\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="34c61598-9e16-46bb-929d-9f2bb9879549" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:27.997906 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:27.997883    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-kj66h\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="e2d8c40c-18cb-4118-af3b-c21c746afea7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:27.999507 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:27.999486    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-h4jrg\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="34c61598-9e16-46bb-929d-9f2bb9879549" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:28.039046 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.039000    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b9x8\" (UniqueName: \"kubernetes.io/projected/952b6ec8-f905-4d6c-905f-9c2844352125-kube-api-access-5b9x8\") pod \"limitador-operator-controller-manager-74495c69f8-rl2wg\" (UID: \"952b6ec8-f905-4d6c-905f-9c2844352125\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rl2wg"
Sep 18 17:58:28.039257 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.039095    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgcw5\" (UniqueName: \"kubernetes.io/projected/d0cbd02b-c258-41ee-8a7b-490bcc517da9-kube-api-access-wgcw5\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9sdzb\" (UID: \"d0cbd02b-c258-41ee-8a7b-490bcc517da9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"
Sep 18 17:58:28.039257 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.039124    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0cbd02b-c258-41ee-8a7b-490bcc517da9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9sdzb\" (UID: \"d0cbd02b-c258-41ee-8a7b-490bcc517da9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"
Sep 18 17:58:28.039505 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.039485    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0cbd02b-c258-41ee-8a7b-490bcc517da9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9sdzb\" (UID: \"d0cbd02b-c258-41ee-8a7b-490bcc517da9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"
Sep 18 17:58:28.043541 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:28.043511    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-kj66h\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="e2d8c40c-18cb-4118-af3b-c21c746afea7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:28.046528 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:28.046501    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-h4jrg\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="34c61598-9e16-46bb-929d-9f2bb9879549" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:28.055680 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.055656    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b9x8\" (UniqueName: \"kubernetes.io/projected/952b6ec8-f905-4d6c-905f-9c2844352125-kube-api-access-5b9x8\") pod \"limitador-operator-controller-manager-74495c69f8-rl2wg\" (UID: \"952b6ec8-f905-4d6c-905f-9c2844352125\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rl2wg"
Sep 18 17:58:28.055761 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.055702    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgcw5\" (UniqueName: \"kubernetes.io/projected/d0cbd02b-c258-41ee-8a7b-490bcc517da9-kube-api-access-wgcw5\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9sdzb\" (UID: \"d0cbd02b-c258-41ee-8a7b-490bcc517da9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"
Sep 18 17:58:28.139867 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.139836    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c61598-9e16-46bb-929d-9f2bb9879549-extensions-socket-volume\") pod \"34c61598-9e16-46bb-929d-9f2bb9879549\" (UID: \"34c61598-9e16-46bb-929d-9f2bb9879549\") "
Sep 18 17:58:28.140033 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.139878    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86bpb\" (UniqueName: \"kubernetes.io/projected/34c61598-9e16-46bb-929d-9f2bb9879549-kube-api-access-86bpb\") pod \"34c61598-9e16-46bb-929d-9f2bb9879549\" (UID: \"34c61598-9e16-46bb-929d-9f2bb9879549\") "
Sep 18 17:58:28.140033 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.139959    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmzks\" (UniqueName: \"kubernetes.io/projected/e2d8c40c-18cb-4118-af3b-c21c746afea7-kube-api-access-tmzks\") pod \"e2d8c40c-18cb-4118-af3b-c21c746afea7\" (UID: \"e2d8c40c-18cb-4118-af3b-c21c746afea7\") "
Sep 18 17:58:28.140400 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.140378    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c61598-9e16-46bb-929d-9f2bb9879549-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "34c61598-9e16-46bb-929d-9f2bb9879549" (UID: "34c61598-9e16-46bb-929d-9f2bb9879549"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 17:58:28.142054 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.142031    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2d8c40c-18cb-4118-af3b-c21c746afea7-kube-api-access-tmzks" (OuterVolumeSpecName: "kube-api-access-tmzks") pod "e2d8c40c-18cb-4118-af3b-c21c746afea7" (UID: "e2d8c40c-18cb-4118-af3b-c21c746afea7"). InnerVolumeSpecName "kube-api-access-tmzks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 17:58:28.142105 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.142060    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34c61598-9e16-46bb-929d-9f2bb9879549-kube-api-access-86bpb" (OuterVolumeSpecName: "kube-api-access-86bpb") pod "34c61598-9e16-46bb-929d-9f2bb9879549" (UID: "34c61598-9e16-46bb-929d-9f2bb9879549"). InnerVolumeSpecName "kube-api-access-86bpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 17:58:28.163352 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.163318    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"
Sep 18 17:58:28.171337 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.171317    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rl2wg"
Sep 18 17:58:28.241314 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.241278    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c61598-9e16-46bb-929d-9f2bb9879549-extensions-socket-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:58:28.241448 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.241321    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86bpb\" (UniqueName: \"kubernetes.io/projected/34c61598-9e16-46bb-929d-9f2bb9879549-kube-api-access-86bpb\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:58:28.241448 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.241352    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmzks\" (UniqueName: \"kubernetes.io/projected/e2d8c40c-18cb-4118-af3b-c21c746afea7-kube-api-access-tmzks\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:58:28.419957 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.419921    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-7mkn8"
Sep 18 17:58:28.460159 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.460071    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-rl2wg"]
Sep 18 17:58:28.464256 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:58:28.462879    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0cbd02b_c258_41ee_8a7b_490bcc517da9.slice/crio-e6f920e27517e6af4dc624ea3bc9b466d8e4be0ece53d64ffea47e10e1324509 WatchSource:0}: Error finding container e6f920e27517e6af4dc624ea3bc9b466d8e4be0ece53d64ffea47e10e1324509: Status 404 returned error can't find the container with id e6f920e27517e6af4dc624ea3bc9b466d8e4be0ece53d64ffea47e10e1324509
Sep 18 17:58:28.464256 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.463853    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"]
Sep 18 17:58:28.476813 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.476761    2568 generic.go:356] "Generic (PLEG): container finished" podID="e2d8c40c-18cb-4118-af3b-c21c746afea7" containerID="d2ffa32d636afdc3bbd6e0481e710a961f90419d67ab30a3541840d3fa0846ac" exitCode=0
Sep 18 17:58:28.476921 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.476837    2568 scope.go:117] "RemoveContainer" containerID="d2ffa32d636afdc3bbd6e0481e710a961f90419d67ab30a3541840d3fa0846ac"
Sep 18 17:58:28.476921 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.476835    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-kj66h"
Sep 18 17:58:28.478818 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.478794    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rl2wg" event={"ID":"952b6ec8-f905-4d6c-905f-9c2844352125","Type":"ContainerStarted","Data":"5b8056104da5e1f0eb9f01e17310d8344bbf69bfec3584d98b2486cf1ab548e8"}
Sep 18 17:58:28.480559 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.480533    2568 generic.go:356] "Generic (PLEG): container finished" podID="34c61598-9e16-46bb-929d-9f2bb9879549" containerID="eee02d397ee91b9e667799f1ad87b3ac8cbcafcf5e5018e5ffdaab3e2af77c91" exitCode=0
Sep 18 17:58:28.480674 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.480590    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-h4jrg"
Sep 18 17:58:28.482735 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.482714    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb" event={"ID":"d0cbd02b-c258-41ee-8a7b-490bcc517da9","Type":"ContainerStarted","Data":"e6f920e27517e6af4dc624ea3bc9b466d8e4be0ece53d64ffea47e10e1324509"}
Sep 18 17:58:28.491134 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.491117    2568 scope.go:117] "RemoveContainer" containerID="d2ffa32d636afdc3bbd6e0481e710a961f90419d67ab30a3541840d3fa0846ac"
Sep 18 17:58:28.491634 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:28.491603    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2ffa32d636afdc3bbd6e0481e710a961f90419d67ab30a3541840d3fa0846ac\": container with ID starting with d2ffa32d636afdc3bbd6e0481e710a961f90419d67ab30a3541840d3fa0846ac not found: ID does not exist" containerID="d2ffa32d636afdc3bbd6e0481e710a961f90419d67ab30a3541840d3fa0846ac"
Sep 18 17:58:28.491734 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.491642    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ffa32d636afdc3bbd6e0481e710a961f90419d67ab30a3541840d3fa0846ac"} err="failed to get container status \"d2ffa32d636afdc3bbd6e0481e710a961f90419d67ab30a3541840d3fa0846ac\": rpc error: code = NotFound desc = could not find container \"d2ffa32d636afdc3bbd6e0481e710a961f90419d67ab30a3541840d3fa0846ac\": container with ID starting with d2ffa32d636afdc3bbd6e0481e710a961f90419d67ab30a3541840d3fa0846ac not found: ID does not exist"
Sep 18 17:58:28.491734 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.491665    2568 scope.go:117] "RemoveContainer" containerID="eee02d397ee91b9e667799f1ad87b3ac8cbcafcf5e5018e5ffdaab3e2af77c91"
Sep 18 17:58:28.505943 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.505921    2568 scope.go:117] "RemoveContainer" containerID="eee02d397ee91b9e667799f1ad87b3ac8cbcafcf5e5018e5ffdaab3e2af77c91"
Sep 18 17:58:28.506295 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:28.506272    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eee02d397ee91b9e667799f1ad87b3ac8cbcafcf5e5018e5ffdaab3e2af77c91\": container with ID starting with eee02d397ee91b9e667799f1ad87b3ac8cbcafcf5e5018e5ffdaab3e2af77c91 not found: ID does not exist" containerID="eee02d397ee91b9e667799f1ad87b3ac8cbcafcf5e5018e5ffdaab3e2af77c91"
Sep 18 17:58:28.506369 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:28.506305    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eee02d397ee91b9e667799f1ad87b3ac8cbcafcf5e5018e5ffdaab3e2af77c91"} err="failed to get container status \"eee02d397ee91b9e667799f1ad87b3ac8cbcafcf5e5018e5ffdaab3e2af77c91\": rpc error: code = NotFound desc = could not find container \"eee02d397ee91b9e667799f1ad87b3ac8cbcafcf5e5018e5ffdaab3e2af77c91\": container with ID starting with eee02d397ee91b9e667799f1ad87b3ac8cbcafcf5e5018e5ffdaab3e2af77c91 not found: ID does not exist"
Sep 18 17:58:29.490535 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.490484    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rl2wg" event={"ID":"952b6ec8-f905-4d6c-905f-9c2844352125","Type":"ContainerStarted","Data":"47183e43066cf5495e4471a5bd8d30c0c457242c241dd83289ac7009879ee851"}
Sep 18 17:58:29.490985 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.490654    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rl2wg"
Sep 18 17:58:29.493036 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.493009    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb" event={"ID":"d0cbd02b-c258-41ee-8a7b-490bcc517da9","Type":"ContainerStarted","Data":"15177dd173b2f668ba9457659581c38835faf3920cfc6bbe365cf653d5eb5de8"}
Sep 18 17:58:29.493143 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.493074    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"
Sep 18 17:58:29.542453 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.542404    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rl2wg" podStartSLOduration=2.542388087 podStartE2EDuration="2.542388087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:58:29.541096815 +0000 UTC m=+513.618900714" watchObservedRunningTime="2026-09-18 17:58:29.542388087 +0000 UTC m=+513.620191986"
Sep 18 17:58:29.570016 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.569956    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb" podStartSLOduration=2.569937642 podStartE2EDuration="2.569937642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:58:29.567705236 +0000 UTC m=+513.645509136" watchObservedRunningTime="2026-09-18 17:58:29.569937642 +0000 UTC m=+513.647741542"
Sep 18 17:58:29.645599 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.645561    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"]
Sep 18 17:58:29.649325 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.649301    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"
Sep 18 17:58:29.672604 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.672575    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"]
Sep 18 17:58:29.754542 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.754451    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x47gh\" (UniqueName: \"kubernetes.io/projected/dc902a56-b126-4d91-9eab-14c0eb02358a-kube-api-access-x47gh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2qqd2\" (UID: \"dc902a56-b126-4d91-9eab-14c0eb02358a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"
Sep 18 17:58:29.754542 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.754489    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc902a56-b126-4d91-9eab-14c0eb02358a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2qqd2\" (UID: \"dc902a56-b126-4d91-9eab-14c0eb02358a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"
Sep 18 17:58:29.855004 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.854964    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x47gh\" (UniqueName: \"kubernetes.io/projected/dc902a56-b126-4d91-9eab-14c0eb02358a-kube-api-access-x47gh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2qqd2\" (UID: \"dc902a56-b126-4d91-9eab-14c0eb02358a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"
Sep 18 17:58:29.855004 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.855015    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc902a56-b126-4d91-9eab-14c0eb02358a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2qqd2\" (UID: \"dc902a56-b126-4d91-9eab-14c0eb02358a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"
Sep 18 17:58:29.855463 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.855445    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc902a56-b126-4d91-9eab-14c0eb02358a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2qqd2\" (UID: \"dc902a56-b126-4d91-9eab-14c0eb02358a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"
Sep 18 17:58:29.868785 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.868754    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x47gh\" (UniqueName: \"kubernetes.io/projected/dc902a56-b126-4d91-9eab-14c0eb02358a-kube-api-access-x47gh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2qqd2\" (UID: \"dc902a56-b126-4d91-9eab-14c0eb02358a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"
Sep 18 17:58:29.959293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:29.959252    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"
Sep 18 17:58:30.045595 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:30.045562    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34c61598-9e16-46bb-929d-9f2bb9879549" path="/var/lib/kubelet/pods/34c61598-9e16-46bb-929d-9f2bb9879549/volumes"
Sep 18 17:58:30.045942 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:30.045923    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2d8c40c-18cb-4118-af3b-c21c746afea7" path="/var/lib/kubelet/pods/e2d8c40c-18cb-4118-af3b-c21c746afea7/volumes"
Sep 18 17:58:30.243589 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:30.243565    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"]
Sep 18 17:58:30.245468 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:58:30.245439    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc902a56_b126_4d91_9eab_14c0eb02358a.slice/crio-166594baa274bfd59335ef0021c82df19b1ac07028cd0bbcfb2a55defdb655d1 WatchSource:0}: Error finding container 166594baa274bfd59335ef0021c82df19b1ac07028cd0bbcfb2a55defdb655d1: Status 404 returned error can't find the container with id 166594baa274bfd59335ef0021c82df19b1ac07028cd0bbcfb2a55defdb655d1
Sep 18 17:58:30.499819 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:30.499779    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2" event={"ID":"dc902a56-b126-4d91-9eab-14c0eb02358a","Type":"ContainerStarted","Data":"e1b23fb41434bf77dccac2f7c9d68bb44968ce6ec6775556d348f3da3ff059e4"}
Sep 18 17:58:30.500294 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:30.499843    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2" event={"ID":"dc902a56-b126-4d91-9eab-14c0eb02358a","Type":"ContainerStarted","Data":"166594baa274bfd59335ef0021c82df19b1ac07028cd0bbcfb2a55defdb655d1"}
Sep 18 17:58:30.500294 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:30.500226    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"
Sep 18 17:58:30.538625 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:30.538521    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2" podStartSLOduration=1.538506993 podStartE2EDuration="1.538506993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:58:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:58:30.537557958 +0000 UTC m=+514.615361859" watchObservedRunningTime="2026-09-18 17:58:30.538506993 +0000 UTC m=+514.616310891"
Sep 18 17:58:40.501924 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:40.501838    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-rl2wg"
Sep 18 17:58:40.502431 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:40.501968    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"
Sep 18 17:58:42.513199 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:42.513147    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"
Sep 18 17:58:42.686153 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:42.686117    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"]
Sep 18 17:58:42.686351 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:42.686325    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb" podUID="d0cbd02b-c258-41ee-8a7b-490bcc517da9" containerName="manager" containerID="cri-o://15177dd173b2f668ba9457659581c38835faf3920cfc6bbe365cf653d5eb5de8" gracePeriod=10
Sep 18 17:58:42.938691 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:42.938665    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"
Sep 18 17:58:42.975873 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:42.975848    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgcw5\" (UniqueName: \"kubernetes.io/projected/d0cbd02b-c258-41ee-8a7b-490bcc517da9-kube-api-access-wgcw5\") pod \"d0cbd02b-c258-41ee-8a7b-490bcc517da9\" (UID: \"d0cbd02b-c258-41ee-8a7b-490bcc517da9\") "
Sep 18 17:58:42.975973 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:42.975881    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0cbd02b-c258-41ee-8a7b-490bcc517da9-extensions-socket-volume\") pod \"d0cbd02b-c258-41ee-8a7b-490bcc517da9\" (UID: \"d0cbd02b-c258-41ee-8a7b-490bcc517da9\") "
Sep 18 17:58:42.976349 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:42.976324    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0cbd02b-c258-41ee-8a7b-490bcc517da9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d0cbd02b-c258-41ee-8a7b-490bcc517da9" (UID: "d0cbd02b-c258-41ee-8a7b-490bcc517da9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 17:58:42.977891 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:42.977874    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0cbd02b-c258-41ee-8a7b-490bcc517da9-kube-api-access-wgcw5" (OuterVolumeSpecName: "kube-api-access-wgcw5") pod "d0cbd02b-c258-41ee-8a7b-490bcc517da9" (UID: "d0cbd02b-c258-41ee-8a7b-490bcc517da9"). InnerVolumeSpecName "kube-api-access-wgcw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 17:58:43.077326 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:43.077299    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgcw5\" (UniqueName: \"kubernetes.io/projected/d0cbd02b-c258-41ee-8a7b-490bcc517da9-kube-api-access-wgcw5\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:58:43.077326 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:43.077322    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0cbd02b-c258-41ee-8a7b-490bcc517da9-extensions-socket-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:58:43.567810 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:43.567774    2568 generic.go:356] "Generic (PLEG): container finished" podID="d0cbd02b-c258-41ee-8a7b-490bcc517da9" containerID="15177dd173b2f668ba9457659581c38835faf3920cfc6bbe365cf653d5eb5de8" exitCode=0
Sep 18 17:58:43.568230 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:43.567829    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"
Sep 18 17:58:43.568230 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:43.567857    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb" event={"ID":"d0cbd02b-c258-41ee-8a7b-490bcc517da9","Type":"ContainerDied","Data":"15177dd173b2f668ba9457659581c38835faf3920cfc6bbe365cf653d5eb5de8"}
Sep 18 17:58:43.568230 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:43.567944    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb" event={"ID":"d0cbd02b-c258-41ee-8a7b-490bcc517da9","Type":"ContainerDied","Data":"e6f920e27517e6af4dc624ea3bc9b466d8e4be0ece53d64ffea47e10e1324509"}
Sep 18 17:58:43.568230 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:43.567971    2568 scope.go:117] "RemoveContainer" containerID="15177dd173b2f668ba9457659581c38835faf3920cfc6bbe365cf653d5eb5de8"
Sep 18 17:58:43.576880 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:43.576863    2568 scope.go:117] "RemoveContainer" containerID="15177dd173b2f668ba9457659581c38835faf3920cfc6bbe365cf653d5eb5de8"
Sep 18 17:58:43.577115 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:58:43.577098    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15177dd173b2f668ba9457659581c38835faf3920cfc6bbe365cf653d5eb5de8\": container with ID starting with 15177dd173b2f668ba9457659581c38835faf3920cfc6bbe365cf653d5eb5de8 not found: ID does not exist" containerID="15177dd173b2f668ba9457659581c38835faf3920cfc6bbe365cf653d5eb5de8"
Sep 18 17:58:43.577165 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:43.577125    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15177dd173b2f668ba9457659581c38835faf3920cfc6bbe365cf653d5eb5de8"} err="failed to get container status \"15177dd173b2f668ba9457659581c38835faf3920cfc6bbe365cf653d5eb5de8\": rpc error: code = NotFound desc = could not find container \"15177dd173b2f668ba9457659581c38835faf3920cfc6bbe365cf653d5eb5de8\": container with ID starting with 15177dd173b2f668ba9457659581c38835faf3920cfc6bbe365cf653d5eb5de8 not found: ID does not exist"
Sep 18 17:58:43.597968 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:43.597935    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"]
Sep 18 17:58:43.605462 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:43.605440    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sdzb"]
Sep 18 17:58:44.044506 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:58:44.044465    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0cbd02b-c258-41ee-8a7b-490bcc517da9" path="/var/lib/kubelet/pods/d0cbd02b-c258-41ee-8a7b-490bcc517da9/volumes"
Sep 18 17:59:01.771240 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:01.771200    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 17:59:01.771857 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:01.771828    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0cbd02b-c258-41ee-8a7b-490bcc517da9" containerName="manager"
Sep 18 17:59:01.771857 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:01.771852    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0cbd02b-c258-41ee-8a7b-490bcc517da9" containerName="manager"
Sep 18 17:59:01.772047 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:01.771984    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0cbd02b-c258-41ee-8a7b-490bcc517da9" containerName="manager"
Sep 18 17:59:01.796797 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:01.796764    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 17:59:01.796947 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:01.796862    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"
Sep 18 17:59:01.799416 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:01.799387    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 18 17:59:01.808810 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:01.808784    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 17:59:01.947688 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:01.947644    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brtt4\" (UniqueName: \"kubernetes.io/projected/36d7e60f-5cd0-4535-a69e-712be4a4a58b-kube-api-access-brtt4\") pod \"limitador-limitador-84994f7ffc-qdzp8\" (UID: \"36d7e60f-5cd0-4535-a69e-712be4a4a58b\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"
Sep 18 17:59:01.947880 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:01.947755    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/36d7e60f-5cd0-4535-a69e-712be4a4a58b-config-file\") pod \"limitador-limitador-84994f7ffc-qdzp8\" (UID: \"36d7e60f-5cd0-4535-a69e-712be4a4a58b\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"
Sep 18 17:59:02.048793 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.048704    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brtt4\" (UniqueName: \"kubernetes.io/projected/36d7e60f-5cd0-4535-a69e-712be4a4a58b-kube-api-access-brtt4\") pod \"limitador-limitador-84994f7ffc-qdzp8\" (UID: \"36d7e60f-5cd0-4535-a69e-712be4a4a58b\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"
Sep 18 17:59:02.048793 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.048772    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/36d7e60f-5cd0-4535-a69e-712be4a4a58b-config-file\") pod \"limitador-limitador-84994f7ffc-qdzp8\" (UID: \"36d7e60f-5cd0-4535-a69e-712be4a4a58b\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"
Sep 18 17:59:02.049469 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.049447    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/36d7e60f-5cd0-4535-a69e-712be4a4a58b-config-file\") pod \"limitador-limitador-84994f7ffc-qdzp8\" (UID: \"36d7e60f-5cd0-4535-a69e-712be4a4a58b\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"
Sep 18 17:59:02.059251 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.059220    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brtt4\" (UniqueName: \"kubernetes.io/projected/36d7e60f-5cd0-4535-a69e-712be4a4a58b-kube-api-access-brtt4\") pod \"limitador-limitador-84994f7ffc-qdzp8\" (UID: \"36d7e60f-5cd0-4535-a69e-712be4a4a58b\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"
Sep 18 17:59:02.106999 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.106964    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"
Sep 18 17:59:02.308397 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.308316    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-6npws"]
Sep 18 17:59:02.312256 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.312233    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-6npws"
Sep 18 17:59:02.315939 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.315914    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qlz5b\""
Sep 18 17:59:02.317956 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.317930    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-6npws"]
Sep 18 17:59:02.387284 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.387256    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 17:59:02.390118 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:59:02.390086    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36d7e60f_5cd0_4535_a69e_712be4a4a58b.slice/crio-c43742aa5bfd7296463781cc5f6dd4ee6e9bd60ef6635f2eed4e6c12fe8a0a29 WatchSource:0}: Error finding container c43742aa5bfd7296463781cc5f6dd4ee6e9bd60ef6635f2eed4e6c12fe8a0a29: Status 404 returned error can't find the container with id c43742aa5bfd7296463781cc5f6dd4ee6e9bd60ef6635f2eed4e6c12fe8a0a29
Sep 18 17:59:02.452349 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.452306    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk8w2\" (UniqueName: \"kubernetes.io/projected/ab8629ab-4e98-411f-9c33-bf3f901a5b45-kube-api-access-nk8w2\") pod \"authorino-59c974bd48-6npws\" (UID: \"ab8629ab-4e98-411f-9c33-bf3f901a5b45\") " pod="kuadrant-system/authorino-59c974bd48-6npws"
Sep 18 17:59:02.553032 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.552994    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk8w2\" (UniqueName: \"kubernetes.io/projected/ab8629ab-4e98-411f-9c33-bf3f901a5b45-kube-api-access-nk8w2\") pod \"authorino-59c974bd48-6npws\" (UID: \"ab8629ab-4e98-411f-9c33-bf3f901a5b45\") " pod="kuadrant-system/authorino-59c974bd48-6npws"
Sep 18 17:59:02.561803 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.561730    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk8w2\" (UniqueName: \"kubernetes.io/projected/ab8629ab-4e98-411f-9c33-bf3f901a5b45-kube-api-access-nk8w2\") pod \"authorino-59c974bd48-6npws\" (UID: \"ab8629ab-4e98-411f-9c33-bf3f901a5b45\") " pod="kuadrant-system/authorino-59c974bd48-6npws"
Sep 18 17:59:02.623807 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.623770    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-6npws"
Sep 18 17:59:02.666697 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.666659    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-qdzp8" event={"ID":"36d7e60f-5cd0-4535-a69e-712be4a4a58b","Type":"ContainerStarted","Data":"c43742aa5bfd7296463781cc5f6dd4ee6e9bd60ef6635f2eed4e6c12fe8a0a29"}
Sep 18 17:59:02.907936 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:02.907913    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-6npws"]
Sep 18 17:59:02.909996 ip-10-0-130-66 kubenswrapper[2568]: W0918 17:59:02.909969    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab8629ab_4e98_411f_9c33_bf3f901a5b45.slice/crio-7fb5c22f2958188a8ccfd31cc3dd9022dc770928727c4a9c6a7f72aed658d722 WatchSource:0}: Error finding container 7fb5c22f2958188a8ccfd31cc3dd9022dc770928727c4a9c6a7f72aed658d722: Status 404 returned error can't find the container with id 7fb5c22f2958188a8ccfd31cc3dd9022dc770928727c4a9c6a7f72aed658d722
Sep 18 17:59:03.673252 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:03.673211    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-6npws" event={"ID":"ab8629ab-4e98-411f-9c33-bf3f901a5b45","Type":"ContainerStarted","Data":"7fb5c22f2958188a8ccfd31cc3dd9022dc770928727c4a9c6a7f72aed658d722"}
Sep 18 17:59:04.679587 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:04.679538    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-qdzp8" event={"ID":"36d7e60f-5cd0-4535-a69e-712be4a4a58b","Type":"ContainerStarted","Data":"4669e83f886f64b0d7d029fdd6af871911a861edda0b26051052c6ddf19c2701"}
Sep 18 17:59:04.680001 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:04.679677    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"
Sep 18 17:59:04.701098 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:04.701044    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-qdzp8" podStartSLOduration=2.195621161 podStartE2EDuration="3.701027963s" totalImagesPullingTime="1.505406802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:59:04.700106869 +0000 UTC m=+548.777910771" watchObservedRunningTime="2026-09-18 17:59:04.701027963 +0000 UTC m=+548.778831862"
Sep 18 17:59:08.702680 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:08.702592    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-6npws" event={"ID":"ab8629ab-4e98-411f-9c33-bf3f901a5b45","Type":"ContainerStarted","Data":"af9e8f90c310ae4c96780cd51fc23f4bb4d90a243f6cca0481cbbc28a53ae046"}
Sep 18 17:59:08.721293 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:08.721245    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-6npws" podStartSLOduration=1.2126195100000001 podStartE2EDuration="6.721230106s" totalImagesPullingTime="5.508610596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:59:08.719632172 +0000 UTC m=+552.797436065" watchObservedRunningTime="2026-09-18 17:59:08.721230106 +0000 UTC m=+552.799034004"
Sep 18 17:59:15.685191 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:15.685160    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"
Sep 18 17:59:31.537708 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:31.537671    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-6npws"]
Sep 18 17:59:31.538221 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:31.537854    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-6npws" podUID="ab8629ab-4e98-411f-9c33-bf3f901a5b45" containerName="authorino" containerID="cri-o://af9e8f90c310ae4c96780cd51fc23f4bb4d90a243f6cca0481cbbc28a53ae046" gracePeriod=30
Sep 18 17:59:31.779337 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:31.779312    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-6npws"
Sep 18 17:59:31.820036 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:31.819953    2568 generic.go:356] "Generic (PLEG): container finished" podID="ab8629ab-4e98-411f-9c33-bf3f901a5b45" containerID="af9e8f90c310ae4c96780cd51fc23f4bb4d90a243f6cca0481cbbc28a53ae046" exitCode=0
Sep 18 17:59:31.820036 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:31.820006    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-6npws"
Sep 18 17:59:31.820285 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:31.820046    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-6npws" event={"ID":"ab8629ab-4e98-411f-9c33-bf3f901a5b45","Type":"ContainerDied","Data":"af9e8f90c310ae4c96780cd51fc23f4bb4d90a243f6cca0481cbbc28a53ae046"}
Sep 18 17:59:31.820285 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:31.820098    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-6npws" event={"ID":"ab8629ab-4e98-411f-9c33-bf3f901a5b45","Type":"ContainerDied","Data":"7fb5c22f2958188a8ccfd31cc3dd9022dc770928727c4a9c6a7f72aed658d722"}
Sep 18 17:59:31.820285 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:31.820123    2568 scope.go:117] "RemoveContainer" containerID="af9e8f90c310ae4c96780cd51fc23f4bb4d90a243f6cca0481cbbc28a53ae046"
Sep 18 17:59:31.828621 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:31.828605    2568 scope.go:117] "RemoveContainer" containerID="af9e8f90c310ae4c96780cd51fc23f4bb4d90a243f6cca0481cbbc28a53ae046"
Sep 18 17:59:31.828899 ip-10-0-130-66 kubenswrapper[2568]: E0918 17:59:31.828880    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af9e8f90c310ae4c96780cd51fc23f4bb4d90a243f6cca0481cbbc28a53ae046\": container with ID starting with af9e8f90c310ae4c96780cd51fc23f4bb4d90a243f6cca0481cbbc28a53ae046 not found: ID does not exist" containerID="af9e8f90c310ae4c96780cd51fc23f4bb4d90a243f6cca0481cbbc28a53ae046"
Sep 18 17:59:31.828946 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:31.828907    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af9e8f90c310ae4c96780cd51fc23f4bb4d90a243f6cca0481cbbc28a53ae046"} err="failed to get container status \"af9e8f90c310ae4c96780cd51fc23f4bb4d90a243f6cca0481cbbc28a53ae046\": rpc error: code = NotFound desc = could not find container \"af9e8f90c310ae4c96780cd51fc23f4bb4d90a243f6cca0481cbbc28a53ae046\": container with ID starting with af9e8f90c310ae4c96780cd51fc23f4bb4d90a243f6cca0481cbbc28a53ae046 not found: ID does not exist"
Sep 18 17:59:31.911149 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:31.911116    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nk8w2\" (UniqueName: \"kubernetes.io/projected/ab8629ab-4e98-411f-9c33-bf3f901a5b45-kube-api-access-nk8w2\") pod \"ab8629ab-4e98-411f-9c33-bf3f901a5b45\" (UID: \"ab8629ab-4e98-411f-9c33-bf3f901a5b45\") "
Sep 18 17:59:31.913244 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:31.913212    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab8629ab-4e98-411f-9c33-bf3f901a5b45-kube-api-access-nk8w2" (OuterVolumeSpecName: "kube-api-access-nk8w2") pod "ab8629ab-4e98-411f-9c33-bf3f901a5b45" (UID: "ab8629ab-4e98-411f-9c33-bf3f901a5b45"). InnerVolumeSpecName "kube-api-access-nk8w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 17:59:32.012006 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:32.011975    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk8w2\" (UniqueName: \"kubernetes.io/projected/ab8629ab-4e98-411f-9c33-bf3f901a5b45-kube-api-access-nk8w2\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 17:59:32.140350 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:32.140271    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-6npws"]
Sep 18 17:59:32.152194 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:32.152148    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-6npws"]
Sep 18 17:59:34.045375 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:34.045343    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab8629ab-4e98-411f-9c33-bf3f901a5b45" path="/var/lib/kubelet/pods/ab8629ab-4e98-411f-9c33-bf3f901a5b45/volumes"
Sep 18 17:59:57.981499 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:57.981469    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 17:59:57.982197 ip-10-0-130-66 kubenswrapper[2568]: I0918 17:59:57.982156    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 18:00:13.906563 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:13.906477    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:00:34.635737 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.635701    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"]
Sep 18 18:00:34.636135 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.636121    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab8629ab-4e98-411f-9c33-bf3f901a5b45" containerName="authorino"
Sep 18 18:00:34.636252 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.636136    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab8629ab-4e98-411f-9c33-bf3f901a5b45" containerName="authorino"
Sep 18 18:00:34.636252 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.636231    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab8629ab-4e98-411f-9c33-bf3f901a5b45" containerName="authorino"
Sep 18 18:00:34.638641 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.638624    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.642458 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.642430    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 18 18:00:34.642589 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.642554    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tq6dk\""
Sep 18 18:00:34.643068 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.643041    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 18 18:00:34.643230 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.643210    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 18 18:00:34.660096 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.660069    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"]
Sep 18 18:00:34.784628 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.784589    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1296604b-af3b-4231-8180-efb81ad8fb9c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.784799 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.784658    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1296604b-af3b-4231-8180-efb81ad8fb9c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.784799 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.784702    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1296604b-af3b-4231-8180-efb81ad8fb9c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.784799 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.784747    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1296604b-af3b-4231-8180-efb81ad8fb9c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.784799 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.784768    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9bkc\" (UniqueName: \"kubernetes.io/projected/1296604b-af3b-4231-8180-efb81ad8fb9c-kube-api-access-k9bkc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.784799 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.784788    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1296604b-af3b-4231-8180-efb81ad8fb9c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.885598 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.885556    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1296604b-af3b-4231-8180-efb81ad8fb9c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.885825 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.885606    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9bkc\" (UniqueName: \"kubernetes.io/projected/1296604b-af3b-4231-8180-efb81ad8fb9c-kube-api-access-k9bkc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.885825 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.885635    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1296604b-af3b-4231-8180-efb81ad8fb9c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.885825 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.885674    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1296604b-af3b-4231-8180-efb81ad8fb9c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.885825 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.885736    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1296604b-af3b-4231-8180-efb81ad8fb9c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.885825 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.885777    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1296604b-af3b-4231-8180-efb81ad8fb9c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.886114 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.886089    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1296604b-af3b-4231-8180-efb81ad8fb9c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.886170 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.886131    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1296604b-af3b-4231-8180-efb81ad8fb9c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.886254 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.886167    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1296604b-af3b-4231-8180-efb81ad8fb9c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.888016 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.887995    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1296604b-af3b-4231-8180-efb81ad8fb9c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.888344 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.888325    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1296604b-af3b-4231-8180-efb81ad8fb9c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.895840 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.895817    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9bkc\" (UniqueName: \"kubernetes.io/projected/1296604b-af3b-4231-8180-efb81ad8fb9c-kube-api-access-k9bkc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4\" (UID: \"1296604b-af3b-4231-8180-efb81ad8fb9c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:34.949029 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:34.948989    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:35.046366 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:35.046329    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:00:35.222149 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:35.222126    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"]
Sep 18 18:00:35.224423 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:00:35.224387    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1296604b_af3b_4231_8180_efb81ad8fb9c.slice/crio-f9c53392ed77557666aa11c8ee93891e6dbeb140ac2a63dc6bcd31db14a17189 WatchSource:0}: Error finding container f9c53392ed77557666aa11c8ee93891e6dbeb140ac2a63dc6bcd31db14a17189: Status 404 returned error can't find the container with id f9c53392ed77557666aa11c8ee93891e6dbeb140ac2a63dc6bcd31db14a17189
Sep 18 18:00:36.143790 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:36.143739    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4" event={"ID":"1296604b-af3b-4231-8180-efb81ad8fb9c","Type":"ContainerStarted","Data":"f9c53392ed77557666aa11c8ee93891e6dbeb140ac2a63dc6bcd31db14a17189"}
Sep 18 18:00:37.150265 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:37.150222    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4" event={"ID":"1296604b-af3b-4231-8180-efb81ad8fb9c","Type":"ContainerStarted","Data":"82c53ee18bdb6f40af0a6953a640eb5dbd27f477ffd5122e5d2bd36c3154df91"}
Sep 18 18:00:37.169895 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:37.169843    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4" podStartSLOduration=2.046531775 podStartE2EDuration="3.169825536s" totalImagesPullingTime="1.123293761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:37.169376568 +0000 UTC m=+641.247180489" watchObservedRunningTime="2026-09-18 18:00:37.169825536 +0000 UTC m=+641.247629434"
Sep 18 18:00:44.950126 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:44.950089    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:44.950631 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:44.950239    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:44.962528 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:44.962502    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:45.199872 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:45.199838    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4"
Sep 18 18:00:46.064110 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:00:46.064070    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:01:06.423440 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:06.423406    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:01:15.014743 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.014707    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"]
Sep 18 18:01:15.017562 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.017529    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.020126 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.020100    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 18 18:01:15.032300 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.032274    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"]
Sep 18 18:01:15.151613 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.151576    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4edcbe87-12d7-4731-858d-eb843d1d1c71-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.151812 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.151621    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xks42\" (UniqueName: \"kubernetes.io/projected/4edcbe87-12d7-4731-858d-eb843d1d1c71-kube-api-access-xks42\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.151812 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.151654    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4edcbe87-12d7-4731-858d-eb843d1d1c71-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.151812 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.151674    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4edcbe87-12d7-4731-858d-eb843d1d1c71-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.151812 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.151745    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4edcbe87-12d7-4731-858d-eb843d1d1c71-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.151812 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.151808    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4edcbe87-12d7-4731-858d-eb843d1d1c71-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.253136 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.253095    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4edcbe87-12d7-4731-858d-eb843d1d1c71-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.253350 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.253207    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4edcbe87-12d7-4731-858d-eb843d1d1c71-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.253350 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.253253    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xks42\" (UniqueName: \"kubernetes.io/projected/4edcbe87-12d7-4731-858d-eb843d1d1c71-kube-api-access-xks42\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.253350 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.253293    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4edcbe87-12d7-4731-858d-eb843d1d1c71-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.253350 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.253322    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4edcbe87-12d7-4731-858d-eb843d1d1c71-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.253568 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.253353    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4edcbe87-12d7-4731-858d-eb843d1d1c71-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.253568 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.253535    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4edcbe87-12d7-4731-858d-eb843d1d1c71-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.253677 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.253573    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4edcbe87-12d7-4731-858d-eb843d1d1c71-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.253731 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.253691    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4edcbe87-12d7-4731-858d-eb843d1d1c71-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.255605 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.255577    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4edcbe87-12d7-4731-858d-eb843d1d1c71-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.255827 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.255809    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4edcbe87-12d7-4731-858d-eb843d1d1c71-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.265074 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.265011    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xks42\" (UniqueName: \"kubernetes.io/projected/4edcbe87-12d7-4731-858d-eb843d1d1c71-kube-api-access-xks42\") pod \"e2e-trlp-test-simulated-kserve-598466548d-k46b9\" (UID: \"4edcbe87-12d7-4731-858d-eb843d1d1c71\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.329138 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.329095    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:15.606110 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.606086    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"]
Sep 18 18:01:15.607721 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:01:15.607692    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4edcbe87_12d7_4731_858d_eb843d1d1c71.slice/crio-39560b441d5f5f1b08e6e8a3e32977cb85c41eb2abbb1673dc3d2e1da231f497 WatchSource:0}: Error finding container 39560b441d5f5f1b08e6e8a3e32977cb85c41eb2abbb1673dc3d2e1da231f497: Status 404 returned error can't find the container with id 39560b441d5f5f1b08e6e8a3e32977cb85c41eb2abbb1673dc3d2e1da231f497
Sep 18 18:01:15.609121 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:15.609103    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:01:16.344270 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:16.344231    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9" event={"ID":"4edcbe87-12d7-4731-858d-eb843d1d1c71","Type":"ContainerStarted","Data":"30494d5aba2751453cbba0f47dfdf6c9582385c26f965807b1f809e94947a5df"}
Sep 18 18:01:16.344923 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:16.344288    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9" event={"ID":"4edcbe87-12d7-4731-858d-eb843d1d1c71","Type":"ContainerStarted","Data":"39560b441d5f5f1b08e6e8a3e32977cb85c41eb2abbb1673dc3d2e1da231f497"}
Sep 18 18:01:16.374681 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:16.369874    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9" podStartSLOduration=1.8089986439999999 podStartE2EDuration="2.369851174s" totalImagesPullingTime="560.85253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:16.367911961 +0000 UTC m=+680.445715860" watchObservedRunningTime="2026-09-18 18:01:16.369851174 +0000 UTC m=+680.447655074"
Sep 18 18:01:16.435668 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:16.435577    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:01:24.594126 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:24.594086    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:01:25.329502 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:25.329468    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:25.329679 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:25.329610    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:25.342100 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:25.342072    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:25.400083 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:25.400056    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-k46b9"
Sep 18 18:01:30.957289 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:30.957254    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"]
Sep 18 18:01:30.960079 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:30.960061    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:30.963539 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:30.963514    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 18 18:01:30.989152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:30.989127    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"]
Sep 18 18:01:30.999524 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:30.999490    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/145faaf8-b9ee-427c-9253-a3b13e561ac7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:30.999524 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:30.999517    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/145faaf8-b9ee-427c-9253-a3b13e561ac7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:30.999735 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:30.999556    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/145faaf8-b9ee-427c-9253-a3b13e561ac7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:30.999735 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:30.999624    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/145faaf8-b9ee-427c-9253-a3b13e561ac7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:30.999735 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:30.999662    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/145faaf8-b9ee-427c-9253-a3b13e561ac7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:30.999735 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:30.999679    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjnv2\" (UniqueName: \"kubernetes.io/projected/145faaf8-b9ee-427c-9253-a3b13e561ac7-kube-api-access-zjnv2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:31.101054 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:31.101011    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/145faaf8-b9ee-427c-9253-a3b13e561ac7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:31.101281 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:31.101071    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/145faaf8-b9ee-427c-9253-a3b13e561ac7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:31.101281 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:31.101096    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjnv2\" (UniqueName: \"kubernetes.io/projected/145faaf8-b9ee-427c-9253-a3b13e561ac7-kube-api-access-zjnv2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:31.101281 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:31.101136    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/145faaf8-b9ee-427c-9253-a3b13e561ac7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:31.101281 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:31.101161    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/145faaf8-b9ee-427c-9253-a3b13e561ac7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:31.101281 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:31.101228    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/145faaf8-b9ee-427c-9253-a3b13e561ac7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:31.101679 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:31.101545    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/145faaf8-b9ee-427c-9253-a3b13e561ac7-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:31.101679 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:31.101599    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/145faaf8-b9ee-427c-9253-a3b13e561ac7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:31.101679 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:31.101651    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/145faaf8-b9ee-427c-9253-a3b13e561ac7-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:31.103393 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:31.103375    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/145faaf8-b9ee-427c-9253-a3b13e561ac7-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:31.103739 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:31.103720    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/145faaf8-b9ee-427c-9253-a3b13e561ac7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:31.110937 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:31.110904    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjnv2\" (UniqueName: \"kubernetes.io/projected/145faaf8-b9ee-427c-9253-a3b13e561ac7-kube-api-access-zjnv2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kz7vn\" (UID: \"145faaf8-b9ee-427c-9253-a3b13e561ac7\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:31.270909 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:31.270820    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:31.549940 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:31.549735    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"]
Sep 18 18:01:31.552677 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:01:31.552642    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod145faaf8_b9ee_427c_9253_a3b13e561ac7.slice/crio-991f5a6b8ef08faabf08fd63a026dc7a0a78723b21dee8cc2e05fa260541aa95 WatchSource:0}: Error finding container 991f5a6b8ef08faabf08fd63a026dc7a0a78723b21dee8cc2e05fa260541aa95: Status 404 returned error can't find the container with id 991f5a6b8ef08faabf08fd63a026dc7a0a78723b21dee8cc2e05fa260541aa95
Sep 18 18:01:32.252543 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:32.252508    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:01:32.424011 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:32.423973    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn" event={"ID":"145faaf8-b9ee-427c-9253-a3b13e561ac7","Type":"ContainerStarted","Data":"3934251e91e667085508cdbdaa9b3d9fd5b09f02dcf87073b76a2d7f4248100b"}
Sep 18 18:01:32.424192 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:32.424033    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn" event={"ID":"145faaf8-b9ee-427c-9253-a3b13e561ac7","Type":"ContainerStarted","Data":"991f5a6b8ef08faabf08fd63a026dc7a0a78723b21dee8cc2e05fa260541aa95"}
Sep 18 18:01:32.446370 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:32.446292    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn" podStartSLOduration=2.270885086 podStartE2EDuration="2.446273766s" totalImagesPullingTime="175.38868ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:32.445511375 +0000 UTC m=+696.523315277" watchObservedRunningTime="2026-09-18 18:01:32.446273766 +0000 UTC m=+696.524077659"
Sep 18 18:01:41.271495 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:41.271410    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:41.271495 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:41.271471    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:41.284034 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:41.284009    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:41.487420 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:41.487388    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kz7vn"
Sep 18 18:01:52.353806 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:01:52.353766    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:03:04.770895 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:04.770862    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d8cf95788-vf7vq"]
Sep 18 18:03:04.774336 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:04.774317    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d8cf95788-vf7vq"
Sep 18 18:03:04.779212 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:04.779188    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 18 18:03:04.782620 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:04.782603    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qlz5b\""
Sep 18 18:03:04.848983 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:04.848954    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d8cf95788-vf7vq"]
Sep 18 18:03:04.859227 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:04.859198    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/135f6fc3-f4fe-4cde-a90e-07ee85823a54-tls-cert\") pod \"authorino-7d8cf95788-vf7vq\" (UID: \"135f6fc3-f4fe-4cde-a90e-07ee85823a54\") " pod="kuadrant-system/authorino-7d8cf95788-vf7vq"
Sep 18 18:03:04.859374 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:04.859323    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkq5b\" (UniqueName: \"kubernetes.io/projected/135f6fc3-f4fe-4cde-a90e-07ee85823a54-kube-api-access-tkq5b\") pod \"authorino-7d8cf95788-vf7vq\" (UID: \"135f6fc3-f4fe-4cde-a90e-07ee85823a54\") " pod="kuadrant-system/authorino-7d8cf95788-vf7vq"
Sep 18 18:03:04.961333 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:04.961302    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkq5b\" (UniqueName: \"kubernetes.io/projected/135f6fc3-f4fe-4cde-a90e-07ee85823a54-kube-api-access-tkq5b\") pod \"authorino-7d8cf95788-vf7vq\" (UID: \"135f6fc3-f4fe-4cde-a90e-07ee85823a54\") " pod="kuadrant-system/authorino-7d8cf95788-vf7vq"
Sep 18 18:03:04.961498 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:04.961365    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/135f6fc3-f4fe-4cde-a90e-07ee85823a54-tls-cert\") pod \"authorino-7d8cf95788-vf7vq\" (UID: \"135f6fc3-f4fe-4cde-a90e-07ee85823a54\") " pod="kuadrant-system/authorino-7d8cf95788-vf7vq"
Sep 18 18:03:04.963699 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:04.963674    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/135f6fc3-f4fe-4cde-a90e-07ee85823a54-tls-cert\") pod \"authorino-7d8cf95788-vf7vq\" (UID: \"135f6fc3-f4fe-4cde-a90e-07ee85823a54\") " pod="kuadrant-system/authorino-7d8cf95788-vf7vq"
Sep 18 18:03:04.982198 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:04.982147    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkq5b\" (UniqueName: \"kubernetes.io/projected/135f6fc3-f4fe-4cde-a90e-07ee85823a54-kube-api-access-tkq5b\") pod \"authorino-7d8cf95788-vf7vq\" (UID: \"135f6fc3-f4fe-4cde-a90e-07ee85823a54\") " pod="kuadrant-system/authorino-7d8cf95788-vf7vq"
Sep 18 18:03:05.083364 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:05.083281    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d8cf95788-vf7vq"
Sep 18 18:03:05.498871 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:05.498840    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d8cf95788-vf7vq"]
Sep 18 18:03:05.502589 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:03:05.502563    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod135f6fc3_f4fe_4cde_a90e_07ee85823a54.slice/crio-c55cea6152df660d9e737295097f7187144f83886bd733caf084daebba13d645 WatchSource:0}: Error finding container c55cea6152df660d9e737295097f7187144f83886bd733caf084daebba13d645: Status 404 returned error can't find the container with id c55cea6152df660d9e737295097f7187144f83886bd733caf084daebba13d645
Sep 18 18:03:05.902622 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:05.902539    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d8cf95788-vf7vq" event={"ID":"135f6fc3-f4fe-4cde-a90e-07ee85823a54","Type":"ContainerStarted","Data":"c55cea6152df660d9e737295097f7187144f83886bd733caf084daebba13d645"}
Sep 18 18:03:06.909609 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:06.909576    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d8cf95788-vf7vq" event={"ID":"135f6fc3-f4fe-4cde-a90e-07ee85823a54","Type":"ContainerStarted","Data":"6a5888e20de32ff180d92e7386b9074dee2a6c65344cf00ebb077de3b08ecf16"}
Sep 18 18:03:06.964854 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:06.964790    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7d8cf95788-vf7vq" podStartSLOduration=2.136218073 podStartE2EDuration="2.964770623s" totalImagesPullingTime="828.55255ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:03:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:03:06.964557514 +0000 UTC m=+791.042361413" watchObservedRunningTime="2026-09-18 18:03:06.964770623 +0000 UTC m=+791.042574526"
Sep 18 18:03:28.577522 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:28.577480    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:03:33.873670 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:33.873638    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"]
Sep 18 18:03:33.877499 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:33.877482    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:33.888169 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:33.888147    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 18:03:33.891434 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:33.891419    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-a39784-openshift-default-dockercfg-b9j7v\""
Sep 18 18:03:33.929029 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:33.928997    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"]
Sep 18 18:03:34.030304 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.030268    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l25wr\" (UniqueName: \"kubernetes.io/projected/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-kube-api-access-l25wr\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.030481 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.030321    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-envoy\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.030481 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.030414    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-credential-socket\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.030481 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.030443    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-workload-certs\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.030481 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.030466    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istiod-ca-cert\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.030657 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.030590    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-token\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.030657 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.030631    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-workload-socket\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.030657 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.030650    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-data\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.030773 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.030683    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-podinfo\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.131905 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.131818    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-envoy\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.131905 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.131880    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-credential-socket\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.132128 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.131910    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-workload-certs\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.132128 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.131942    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istiod-ca-cert\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.132128 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.131993    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-token\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.132128 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.132025    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-workload-socket\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.132128 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.132045    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-data\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.132128 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.132092    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-podinfo\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.132481 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.132158    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l25wr\" (UniqueName: \"kubernetes.io/projected/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-kube-api-access-l25wr\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.132481 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.132396    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-credential-socket\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.132481 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.132400    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-workload-certs\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.132481 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.132441    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-workload-socket\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.132685 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.132638    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-data\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.133038 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.133016    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istiod-ca-cert\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.134201 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.134167    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-envoy\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.134651 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.134625    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-podinfo\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.141614 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.141595    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-token\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.141814 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.141796    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l25wr\" (UniqueName: \"kubernetes.io/projected/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-kube-api-access-l25wr\") pod \"e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") " pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.187237 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.187209    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:34.474523 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.474401    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"]
Sep 18 18:03:34.476725 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:03:34.476692    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a88d247_cfeb_40b7_8b1c_8bfd06fd7209.slice/crio-69a1493cfdf34eeba26af88d604fbca7f562ce0a25227394cf6d21158b3d32b6 WatchSource:0}: Error finding container 69a1493cfdf34eeba26af88d604fbca7f562ce0a25227394cf6d21158b3d32b6: Status 404 returned error can't find the container with id 69a1493cfdf34eeba26af88d604fbca7f562ce0a25227394cf6d21158b3d32b6
Sep 18 18:03:34.835169 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.835087    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"]
Sep 18 18:03:34.840005 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.839983    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:34.859451 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.859265    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-609aba-serving-cert\""
Sep 18 18:03:34.859451 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.859374    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 18:03:34.859641 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.859523    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 18:03:34.859754 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.859740    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tlsbx\""
Sep 18 18:03:34.860157 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.860137    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:03:34.877231 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.877209    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"]
Sep 18 18:03:34.939890 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.939858    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:34.939890 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.939890    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-metrics-certs\") pod \"maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:34.940134 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.940023    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8tst\" (UniqueName: \"kubernetes.io/projected/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-kube-api-access-w8tst\") pod \"maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:34.940134 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:34.940104    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:35.041507 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:35.041478    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:35.041692 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:35.041519    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:35.041692 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:35.041536    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-metrics-certs\") pod \"maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:35.041692 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:35.041600    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8tst\" (UniqueName: \"kubernetes.io/projected/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-kube-api-access-w8tst\") pod \"maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:35.041692 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:03:35.041656    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-serving-cert: secret "maas-api-e2e-worker-w0-609aba-serving-cert" not found
Sep 18 18:03:35.041897 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:03:35.041723    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-maas-api-tls podName:f1fec7ec-ea60-4ae4-8ac6-4748b259986b nodeName:}" failed. No retries permitted until 2026-09-18 18:03:35.541700471 +0000 UTC m=+819.619504375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-maas-api-tls") pod "maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2" (UID: "f1fec7ec-ea60-4ae4-8ac6-4748b259986b") : secret "maas-api-e2e-worker-w0-609aba-serving-cert" not found
Sep 18 18:03:35.042463 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:35.042444    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:35.043963 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:35.043945    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-metrics-certs\") pod \"maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:35.054648 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:35.054618    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh" event={"ID":"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209","Type":"ContainerStarted","Data":"69a1493cfdf34eeba26af88d604fbca7f562ce0a25227394cf6d21158b3d32b6"}
Sep 18 18:03:35.063407 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:35.063386    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8tst\" (UniqueName: \"kubernetes.io/projected/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-kube-api-access-w8tst\") pod \"maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:35.547118 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:35.547083    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:35.550270 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:35.550113    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:35.751403 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:35.751368    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:36.080598 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:36.080575    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"]
Sep 18 18:03:36.082906 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:03:36.082884    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1fec7ec_ea60_4ae4_8ac6_4748b259986b.slice/crio-1a73b9eef51168290c9f50950df62bbcda9678d5215b6e6e1c20dd80952e59c1 WatchSource:0}: Error finding container 1a73b9eef51168290c9f50950df62bbcda9678d5215b6e6e1c20dd80952e59c1: Status 404 returned error can't find the container with id 1a73b9eef51168290c9f50950df62bbcda9678d5215b6e6e1c20dd80952e59c1
Sep 18 18:03:37.072945 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:37.072911    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2" event={"ID":"f1fec7ec-ea60-4ae4-8ac6-4748b259986b","Type":"ContainerStarted","Data":"1a73b9eef51168290c9f50950df62bbcda9678d5215b6e6e1c20dd80952e59c1"}
Sep 18 18:03:39.548499 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:39.548450    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:03:39.548885 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:39.548538    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:03:39.548885 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:39.548586    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:03:40.092373 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:40.092338    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh" event={"ID":"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209","Type":"ContainerStarted","Data":"5b5a0e68f662772861cbdb229b9a653c5b37a424a5fd860f441280a0487187c1"}
Sep 18 18:03:40.093852 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:40.093823    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2" event={"ID":"f1fec7ec-ea60-4ae4-8ac6-4748b259986b","Type":"ContainerStarted","Data":"f410f25f04836e117f9a9ae5a9305bc1deb001f497d40be2db65b9527d0cb62b"}
Sep 18 18:03:40.093982 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:40.093872    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:40.188002 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:40.187965    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:40.193082 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:40.193058    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:40.206692 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:40.206646    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh" podStartSLOduration=2.137320994 podStartE2EDuration="7.20663039s" totalImagesPullingTime="5.069309396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:03:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:03:40.193709342 +0000 UTC m=+824.271513240" watchObservedRunningTime="2026-09-18 18:03:40.20663039 +0000 UTC m=+824.284434289"
Sep 18 18:03:40.323590 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:40.323538    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2" podStartSLOduration=2.808856196 podStartE2EDuration="6.323520449s" totalImagesPullingTime="3.514664253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:03:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:03:40.323188292 +0000 UTC m=+824.400992183" watchObservedRunningTime="2026-09-18 18:03:40.323520449 +0000 UTC m=+824.401324349"
Sep 18 18:03:41.099136 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:41.099098    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:41.100196 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:41.100155    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:03:44.138847 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:44.138803    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:03:46.104832 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.104803    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:03:46.767154 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.767117    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"]
Sep 18 18:03:46.770547 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.770524    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:46.773893 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.773865    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-63da2c01-serving-cert\""
Sep 18 18:03:46.794379 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.794353    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"]
Sep 18 18:03:46.853315 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.853287    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1592a127-7f26-4b3f-a792-4554cc41aba4-metrics-certs\") pod \"maas-api-e2e-ait-63da2c01-6876fc7865-sv889\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:46.853488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.853324    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1592a127-7f26-4b3f-a792-4554cc41aba4-openshift-service-ca\") pod \"maas-api-e2e-ait-63da2c01-6876fc7865-sv889\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:46.853488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.853346    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-272kb\" (UniqueName: \"kubernetes.io/projected/1592a127-7f26-4b3f-a792-4554cc41aba4-kube-api-access-272kb\") pod \"maas-api-e2e-ait-63da2c01-6876fc7865-sv889\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:46.853488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.853455    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1592a127-7f26-4b3f-a792-4554cc41aba4-maas-api-tls\") pod \"maas-api-e2e-ait-63da2c01-6876fc7865-sv889\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:46.954742 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.954712    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1592a127-7f26-4b3f-a792-4554cc41aba4-maas-api-tls\") pod \"maas-api-e2e-ait-63da2c01-6876fc7865-sv889\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:46.954908 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.954788    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1592a127-7f26-4b3f-a792-4554cc41aba4-metrics-certs\") pod \"maas-api-e2e-ait-63da2c01-6876fc7865-sv889\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:46.954908 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.954810    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1592a127-7f26-4b3f-a792-4554cc41aba4-openshift-service-ca\") pod \"maas-api-e2e-ait-63da2c01-6876fc7865-sv889\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:46.954908 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.954832    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-272kb\" (UniqueName: \"kubernetes.io/projected/1592a127-7f26-4b3f-a792-4554cc41aba4-kube-api-access-272kb\") pod \"maas-api-e2e-ait-63da2c01-6876fc7865-sv889\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:46.954908 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:03:46.954877    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-serving-cert: secret "maas-api-e2e-ait-63da2c01-serving-cert" not found
Sep 18 18:03:46.955122 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:03:46.954944    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1592a127-7f26-4b3f-a792-4554cc41aba4-maas-api-tls podName:1592a127-7f26-4b3f-a792-4554cc41aba4 nodeName:}" failed. No retries permitted until 2026-09-18 18:03:47.454925538 +0000 UTC m=+831.532729429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1592a127-7f26-4b3f-a792-4554cc41aba4-maas-api-tls") pod "maas-api-e2e-ait-63da2c01-6876fc7865-sv889" (UID: "1592a127-7f26-4b3f-a792-4554cc41aba4") : secret "maas-api-e2e-ait-63da2c01-serving-cert" not found
Sep 18 18:03:46.955496 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.955476    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1592a127-7f26-4b3f-a792-4554cc41aba4-openshift-service-ca\") pod \"maas-api-e2e-ait-63da2c01-6876fc7865-sv889\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:46.957404 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.957374    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1592a127-7f26-4b3f-a792-4554cc41aba4-metrics-certs\") pod \"maas-api-e2e-ait-63da2c01-6876fc7865-sv889\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:46.967821 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:46.967798    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-272kb\" (UniqueName: \"kubernetes.io/projected/1592a127-7f26-4b3f-a792-4554cc41aba4-kube-api-access-272kb\") pod \"maas-api-e2e-ait-63da2c01-6876fc7865-sv889\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:47.459983 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:47.459943    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1592a127-7f26-4b3f-a792-4554cc41aba4-maas-api-tls\") pod \"maas-api-e2e-ait-63da2c01-6876fc7865-sv889\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:47.463243 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:47.463217    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1592a127-7f26-4b3f-a792-4554cc41aba4-maas-api-tls\") pod \"maas-api-e2e-ait-63da2c01-6876fc7865-sv889\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:47.682681 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:47.682647    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:48.005232 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:03:48.005206    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1592a127_7f26_4b3f_a792_4554cc41aba4.slice/crio-33d585c7207991ebac9865e21760eb54cb848c9efc1d1e81f075e1486e90aa0b WatchSource:0}: Error finding container 33d585c7207991ebac9865e21760eb54cb848c9efc1d1e81f075e1486e90aa0b: Status 404 returned error can't find the container with id 33d585c7207991ebac9865e21760eb54cb848c9efc1d1e81f075e1486e90aa0b
Sep 18 18:03:48.008836 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:48.008810    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"]
Sep 18 18:03:48.135341 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:48.135307    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889" event={"ID":"1592a127-7f26-4b3f-a792-4554cc41aba4","Type":"ContainerStarted","Data":"33d585c7207991ebac9865e21760eb54cb848c9efc1d1e81f075e1486e90aa0b"}
Sep 18 18:03:48.847401 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:48.847368    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"]
Sep 18 18:03:48.850936 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:48.850920    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"
Sep 18 18:03:48.857230 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:48.857204    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-63da2c01\""
Sep 18 18:03:48.857379 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:48.857232    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-63da2c01-dockercfg-wmng6\""
Sep 18 18:03:48.871087 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:48.871062    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"]
Sep 18 18:03:48.873297 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:48.873272    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czwkl\" (UniqueName: \"kubernetes.io/projected/7921d9ed-b5d1-4442-a278-ba77f3040f00-kube-api-access-czwkl\") pod \"payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg\" (UID: \"7921d9ed-b5d1-4442-a278-ba77f3040f00\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"
Sep 18 18:03:48.873420 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:48.873404    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7921d9ed-b5d1-4442-a278-ba77f3040f00-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg\" (UID: \"7921d9ed-b5d1-4442-a278-ba77f3040f00\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"
Sep 18 18:03:48.974662 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:48.974626    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7921d9ed-b5d1-4442-a278-ba77f3040f00-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg\" (UID: \"7921d9ed-b5d1-4442-a278-ba77f3040f00\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"
Sep 18 18:03:48.974835 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:48.974686    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czwkl\" (UniqueName: \"kubernetes.io/projected/7921d9ed-b5d1-4442-a278-ba77f3040f00-kube-api-access-czwkl\") pod \"payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg\" (UID: \"7921d9ed-b5d1-4442-a278-ba77f3040f00\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"
Sep 18 18:03:48.975353 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:48.975335    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7921d9ed-b5d1-4442-a278-ba77f3040f00-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg\" (UID: \"7921d9ed-b5d1-4442-a278-ba77f3040f00\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"
Sep 18 18:03:48.985502 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:48.985480    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czwkl\" (UniqueName: \"kubernetes.io/projected/7921d9ed-b5d1-4442-a278-ba77f3040f00-kube-api-access-czwkl\") pod \"payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg\" (UID: \"7921d9ed-b5d1-4442-a278-ba77f3040f00\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"
Sep 18 18:03:49.142120 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:49.142042    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889" event={"ID":"1592a127-7f26-4b3f-a792-4554cc41aba4","Type":"ContainerStarted","Data":"e8bf4f77fe09cb7513e2816a08667ff0efe1e21a398e5cd46e4dec1c3f4a52f9"}
Sep 18 18:03:49.142300 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:49.142148    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:49.162678 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:49.162642    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"
Sep 18 18:03:49.451167 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:49.451048    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889" podStartSLOduration=2.4444124 podStartE2EDuration="3.451029582s" totalImagesPullingTime="1.006617182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:03:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:03:49.209142252 +0000 UTC m=+833.286946151" watchObservedRunningTime="2026-09-18 18:03:49.451029582 +0000 UTC m=+833.528833481"
Sep 18 18:03:49.451995 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:49.451977    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"]
Sep 18 18:03:49.454006 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:03:49.453981    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7921d9ed_b5d1_4442_a278_ba77f3040f00.slice/crio-26a52768457edb97048e0265016987ae3baeb1362e6a4b32326d23d61dc65d57 WatchSource:0}: Error finding container 26a52768457edb97048e0265016987ae3baeb1362e6a4b32326d23d61dc65d57: Status 404 returned error can't find the container with id 26a52768457edb97048e0265016987ae3baeb1362e6a4b32326d23d61dc65d57
Sep 18 18:03:50.149443 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.149402    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg" event={"ID":"7921d9ed-b5d1-4442-a278-ba77f3040f00","Type":"ContainerStarted","Data":"26a52768457edb97048e0265016987ae3baeb1362e6a4b32326d23d61dc65d57"}
Sep 18 18:03:50.590568 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.590524    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"]
Sep 18 18:03:50.594797 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.594775    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"
Sep 18 18:03:50.598838 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.598810    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-a39784\""
Sep 18 18:03:50.599053 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.598937    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-a39784-dockercfg-8xwf2\""
Sep 18 18:03:50.606428 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.606406    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"]
Sep 18 18:03:50.685712 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.685688    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"]
Sep 18 18:03:50.692725 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.692707    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"
Sep 18 18:03:50.693150 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.693130    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/947b2e8f-8ba2-4912-b0f2-f59b76e3f62e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8\" (UID: \"947b2e8f-8ba2-4912-b0f2-f59b76e3f62e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"
Sep 18 18:03:50.693260 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.693205    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c9cz\" (UniqueName: \"kubernetes.io/projected/947b2e8f-8ba2-4912-b0f2-f59b76e3f62e-kube-api-access-9c9cz\") pod \"payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8\" (UID: \"947b2e8f-8ba2-4912-b0f2-f59b76e3f62e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"
Sep 18 18:03:50.708437 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.708411    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"]
Sep 18 18:03:50.793791 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.793758    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be0f070a-70fd-4268-a019-a45f147315e8-plugins-config-volume\") pod \"payload-processing-e2e-autores-a39784-7f855f565c-l9hsr\" (UID: \"be0f070a-70fd-4268-a019-a45f147315e8\") " pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"
Sep 18 18:03:50.793946 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.793808    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c9cz\" (UniqueName: \"kubernetes.io/projected/947b2e8f-8ba2-4912-b0f2-f59b76e3f62e-kube-api-access-9c9cz\") pod \"payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8\" (UID: \"947b2e8f-8ba2-4912-b0f2-f59b76e3f62e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"
Sep 18 18:03:50.793985 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.793938    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgp8q\" (UniqueName: \"kubernetes.io/projected/be0f070a-70fd-4268-a019-a45f147315e8-kube-api-access-cgp8q\") pod \"payload-processing-e2e-autores-a39784-7f855f565c-l9hsr\" (UID: \"be0f070a-70fd-4268-a019-a45f147315e8\") " pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"
Sep 18 18:03:50.794037 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.794021    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/947b2e8f-8ba2-4912-b0f2-f59b76e3f62e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8\" (UID: \"947b2e8f-8ba2-4912-b0f2-f59b76e3f62e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"
Sep 18 18:03:50.794653 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.794635    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/947b2e8f-8ba2-4912-b0f2-f59b76e3f62e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8\" (UID: \"947b2e8f-8ba2-4912-b0f2-f59b76e3f62e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"
Sep 18 18:03:50.804599 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.804573    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c9cz\" (UniqueName: \"kubernetes.io/projected/947b2e8f-8ba2-4912-b0f2-f59b76e3f62e-kube-api-access-9c9cz\") pod \"payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8\" (UID: \"947b2e8f-8ba2-4912-b0f2-f59b76e3f62e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"
Sep 18 18:03:50.895233 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.895123    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be0f070a-70fd-4268-a019-a45f147315e8-plugins-config-volume\") pod \"payload-processing-e2e-autores-a39784-7f855f565c-l9hsr\" (UID: \"be0f070a-70fd-4268-a019-a45f147315e8\") " pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"
Sep 18 18:03:50.895376 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.895283    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgp8q\" (UniqueName: \"kubernetes.io/projected/be0f070a-70fd-4268-a019-a45f147315e8-kube-api-access-cgp8q\") pod \"payload-processing-e2e-autores-a39784-7f855f565c-l9hsr\" (UID: \"be0f070a-70fd-4268-a019-a45f147315e8\") " pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"
Sep 18 18:03:50.895870 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.895849    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be0f070a-70fd-4268-a019-a45f147315e8-plugins-config-volume\") pod \"payload-processing-e2e-autores-a39784-7f855f565c-l9hsr\" (UID: \"be0f070a-70fd-4268-a019-a45f147315e8\") " pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"
Sep 18 18:03:50.905671 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.905649    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgp8q\" (UniqueName: \"kubernetes.io/projected/be0f070a-70fd-4268-a019-a45f147315e8-kube-api-access-cgp8q\") pod \"payload-processing-e2e-autores-a39784-7f855f565c-l9hsr\" (UID: \"be0f070a-70fd-4268-a019-a45f147315e8\") " pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"
Sep 18 18:03:50.907316 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:50.907297    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"
Sep 18 18:03:51.005787 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:51.005760    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"
Sep 18 18:03:51.157385 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:51.156433    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg" event={"ID":"7921d9ed-b5d1-4442-a278-ba77f3040f00","Type":"ContainerStarted","Data":"4827e58d81c9d4977459a2849aeb8d70ff7a5401065f8788ab7a2a5eadc78512"}
Sep 18 18:03:51.157385 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:51.157272    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"
Sep 18 18:03:51.186655 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:51.186586    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg" podStartSLOduration=2.022835194 podStartE2EDuration="3.186567552s" totalImagesPullingTime="1.163732358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:03:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:03:51.184721141 +0000 UTC m=+835.262525040" watchObservedRunningTime="2026-09-18 18:03:51.186567552 +0000 UTC m=+835.264371452"
Sep 18 18:03:51.192798 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:51.192776    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"]
Sep 18 18:03:51.194895 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:03:51.194869    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod947b2e8f_8ba2_4912_b0f2_f59b76e3f62e.slice/crio-8f45acabebe24591b3f5499e873f5bff3247671a7702ea207ae6b8dd14648773 WatchSource:0}: Error finding container 8f45acabebe24591b3f5499e873f5bff3247671a7702ea207ae6b8dd14648773: Status 404 returned error can't find the container with id 8f45acabebe24591b3f5499e873f5bff3247671a7702ea207ae6b8dd14648773
Sep 18 18:03:51.296629 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:51.296604    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"]
Sep 18 18:03:51.297901 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:03:51.297876    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe0f070a_70fd_4268_a019_a45f147315e8.slice/crio-71eb1899ca9cb9b726bc1f9b9bcc3ee03674585dc1f03662371076c42109afbe WatchSource:0}: Error finding container 71eb1899ca9cb9b726bc1f9b9bcc3ee03674585dc1f03662371076c42109afbe: Status 404 returned error can't find the container with id 71eb1899ca9cb9b726bc1f9b9bcc3ee03674585dc1f03662371076c42109afbe
Sep 18 18:03:52.165757 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:52.165726    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr" event={"ID":"be0f070a-70fd-4268-a019-a45f147315e8","Type":"ContainerStarted","Data":"afa1b06b50987927f3dda4be8f56702ab79478416f6eca43dce8082448252470"}
Sep 18 18:03:52.166241 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:52.165822    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"
Sep 18 18:03:52.166241 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:52.165865    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr" event={"ID":"be0f070a-70fd-4268-a019-a45f147315e8","Type":"ContainerStarted","Data":"71eb1899ca9cb9b726bc1f9b9bcc3ee03674585dc1f03662371076c42109afbe"}
Sep 18 18:03:52.167438 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:52.167411    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8" event={"ID":"947b2e8f-8ba2-4912-b0f2-f59b76e3f62e","Type":"ContainerStarted","Data":"bcc1dc0fc21b0b229c096f0702c90ccd5886101d40f9224c8e144a07e3e8b102"}
Sep 18 18:03:52.167539 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:52.167463    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8" event={"ID":"947b2e8f-8ba2-4912-b0f2-f59b76e3f62e","Type":"ContainerStarted","Data":"8f45acabebe24591b3f5499e873f5bff3247671a7702ea207ae6b8dd14648773"}
Sep 18 18:03:52.167617 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:52.167537    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"
Sep 18 18:03:52.208359 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:52.208303    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr" podStartSLOduration=2.208287943 podStartE2EDuration="2.208287943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:03:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:03:52.205825079 +0000 UTC m=+836.283628977" watchObservedRunningTime="2026-09-18 18:03:52.208287943 +0000 UTC m=+836.286091839"
Sep 18 18:03:52.262210 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:52.262148    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8" podStartSLOduration=2.26213153 podStartE2EDuration="2.26213153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:03:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:03:52.258482286 +0000 UTC m=+836.336286184" watchObservedRunningTime="2026-09-18 18:03:52.26213153 +0000 UTC m=+836.339935429"
Sep 18 18:03:55.155231 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:55.155196    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:03:55.833362 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:55.833328    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"]
Sep 18 18:03:55.837140 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:55.837123    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"
Sep 18 18:03:55.842068 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:55.842048    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ee3049-dockercfg-wvjqq\""
Sep 18 18:03:55.843121 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:55.843105    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ee3049\""
Sep 18 18:03:55.873199 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:55.870903    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"]
Sep 18 18:03:55.943857 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:55.943830    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzjzb\" (UniqueName: \"kubernetes.io/projected/f8923b82-671a-40a2-a5ea-86c8503e4683-kube-api-access-qzjzb\") pod \"payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk\" (UID: \"f8923b82-671a-40a2-a5ea-86c8503e4683\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"
Sep 18 18:03:55.944016 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:55.943914    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8923b82-671a-40a2-a5ea-86c8503e4683-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk\" (UID: \"f8923b82-671a-40a2-a5ea-86c8503e4683\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"
Sep 18 18:03:56.044678 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:56.044647    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8923b82-671a-40a2-a5ea-86c8503e4683-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk\" (UID: \"f8923b82-671a-40a2-a5ea-86c8503e4683\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"
Sep 18 18:03:56.044834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:56.044712    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzjzb\" (UniqueName: \"kubernetes.io/projected/f8923b82-671a-40a2-a5ea-86c8503e4683-kube-api-access-qzjzb\") pod \"payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk\" (UID: \"f8923b82-671a-40a2-a5ea-86c8503e4683\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"
Sep 18 18:03:56.045610 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:56.045586    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8923b82-671a-40a2-a5ea-86c8503e4683-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk\" (UID: \"f8923b82-671a-40a2-a5ea-86c8503e4683\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"
Sep 18 18:03:56.071721 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:56.071658    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzjzb\" (UniqueName: \"kubernetes.io/projected/f8923b82-671a-40a2-a5ea-86c8503e4683-kube-api-access-qzjzb\") pod \"payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk\" (UID: \"f8923b82-671a-40a2-a5ea-86c8503e4683\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"
Sep 18 18:03:56.148075 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:56.148003    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"
Sep 18 18:03:56.473974 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:56.460906    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"]
Sep 18 18:03:57.205898 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:57.205528    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk" event={"ID":"f8923b82-671a-40a2-a5ea-86c8503e4683","Type":"ContainerStarted","Data":"ccf17fbce2f9f9e0b4f1fe9501eeac59976a20bdfa2802b8ff62949d96db3116"}
Sep 18 18:03:57.205898 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:57.205699    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"
Sep 18 18:03:57.205898 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:57.205735    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk" event={"ID":"f8923b82-671a-40a2-a5ea-86c8503e4683","Type":"ContainerStarted","Data":"f41dc94295c80971b6b4f571bb848b1d4a74a0e3ddd3da57ecef978e5e2d923f"}
Sep 18 18:03:57.237366 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:03:57.237301    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk" podStartSLOduration=2.23728225 podStartE2EDuration="2.23728225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:03:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:03:57.237087968 +0000 UTC m=+841.314891868" watchObservedRunningTime="2026-09-18 18:03:57.23728225 +0000 UTC m=+841.315086150"
Sep 18 18:04:00.255677 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:00.255645    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:04:03.175428 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:03.175387    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"
Sep 18 18:04:03.175910 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:03.175589    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"
Sep 18 18:04:03.175910 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:03.175691    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"
Sep 18 18:04:03.244818 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:03.244787    2568 generic.go:356] "Generic (PLEG): container finished" podID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" containerID="5b5a0e68f662772861cbdb229b9a653c5b37a424a5fd860f441280a0487187c1" exitCode=0
Sep 18 18:04:03.245009 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:03.244870    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh" event={"ID":"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209","Type":"ContainerDied","Data":"5b5a0e68f662772861cbdb229b9a653c5b37a424a5fd860f441280a0487187c1"}
Sep 18 18:04:03.245429 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:03.245408    2568 scope.go:117] "RemoveContainer" containerID="5b5a0e68f662772861cbdb229b9a653c5b37a424a5fd860f441280a0487187c1"
Sep 18 18:04:03.246686 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:03.246657    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:04:03.246793 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:03.246727    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:04:03.246793 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:03.246761    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:04:04.252039 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:04.252006    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh" event={"ID":"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209","Type":"ContainerStarted","Data":"00fcfc841b20b440f6eec3bf393b57bbe1a14f650a1a7803cc615c134edb35b1"}
Sep 18 18:04:05.188412 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:05.188371    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:04:05.188749 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:05.188727    2568 patch_prober.go:28] interesting pod/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 18 18:04:05.188812 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:05.188770    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh" podUID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 18 18:04:06.188673 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:06.188630    2568 patch_prober.go:28] interesting pod/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 18 18:04:06.189103 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:06.188705    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh" podUID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 18 18:04:07.187686 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:07.187647    2568 patch_prober.go:28] interesting pod/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 18 18:04:07.187870 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:07.187708    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh" podUID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 18 18:04:08.188643 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:08.188606    2568 patch_prober.go:28] interesting pod/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 18 18:04:08.189040 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:08.188676    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh" podUID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 18 18:04:08.213934 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:08.213908    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"
Sep 18 18:04:09.187942 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:09.187902    2568 patch_prober.go:28] interesting pod/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 18 18:04:09.188200 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:09.187972    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh" podUID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 18 18:04:10.192489 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:10.192453    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:04:10.192930 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:10.192727    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:04:10.193570 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:10.193547    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:04:13.881347 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:13.881316    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"]
Sep 18 18:04:13.881740 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:13.881516    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889" podUID="1592a127-7f26-4b3f-a792-4554cc41aba4" containerName="maas-api" containerID="cri-o://e8bf4f77fe09cb7513e2816a08667ff0efe1e21a398e5cd46e4dec1c3f4a52f9" gracePeriod=30
Sep 18 18:04:13.996038 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:13.996011    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"]
Sep 18 18:04:13.996269 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:13.996231    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg" podUID="7921d9ed-b5d1-4442-a278-ba77f3040f00" containerName="payload-pre-processing" containerID="cri-o://4827e58d81c9d4977459a2849aeb8d70ff7a5401065f8788ab7a2a5eadc78512" gracePeriod=30
Sep 18 18:04:14.138497 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.138433    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:04:14.240103 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.240068    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-272kb\" (UniqueName: \"kubernetes.io/projected/1592a127-7f26-4b3f-a792-4554cc41aba4-kube-api-access-272kb\") pod \"1592a127-7f26-4b3f-a792-4554cc41aba4\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") "
Sep 18 18:04:14.240295 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.240127    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1592a127-7f26-4b3f-a792-4554cc41aba4-maas-api-tls\") pod \"1592a127-7f26-4b3f-a792-4554cc41aba4\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") "
Sep 18 18:04:14.240295 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.240154    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1592a127-7f26-4b3f-a792-4554cc41aba4-openshift-service-ca\") pod \"1592a127-7f26-4b3f-a792-4554cc41aba4\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") "
Sep 18 18:04:14.240425 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.240402    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1592a127-7f26-4b3f-a792-4554cc41aba4-metrics-certs\") pod \"1592a127-7f26-4b3f-a792-4554cc41aba4\" (UID: \"1592a127-7f26-4b3f-a792-4554cc41aba4\") "
Sep 18 18:04:14.240565 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.240534    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1592a127-7f26-4b3f-a792-4554cc41aba4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1592a127-7f26-4b3f-a792-4554cc41aba4" (UID: "1592a127-7f26-4b3f-a792-4554cc41aba4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:04:14.240879 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.240845    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1592a127-7f26-4b3f-a792-4554cc41aba4-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:04:14.242890 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.242858    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1592a127-7f26-4b3f-a792-4554cc41aba4-kube-api-access-272kb" (OuterVolumeSpecName: "kube-api-access-272kb") pod "1592a127-7f26-4b3f-a792-4554cc41aba4" (UID: "1592a127-7f26-4b3f-a792-4554cc41aba4"). InnerVolumeSpecName "kube-api-access-272kb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:04:14.242990 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.242916    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1592a127-7f26-4b3f-a792-4554cc41aba4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1592a127-7f26-4b3f-a792-4554cc41aba4" (UID: "1592a127-7f26-4b3f-a792-4554cc41aba4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:04:14.242990 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.242947    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1592a127-7f26-4b3f-a792-4554cc41aba4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1592a127-7f26-4b3f-a792-4554cc41aba4" (UID: "1592a127-7f26-4b3f-a792-4554cc41aba4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:04:14.248463 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.248445    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"
Sep 18 18:04:14.333025 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.325726    2568 generic.go:356] "Generic (PLEG): container finished" podID="1592a127-7f26-4b3f-a792-4554cc41aba4" containerID="e8bf4f77fe09cb7513e2816a08667ff0efe1e21a398e5cd46e4dec1c3f4a52f9" exitCode=0
Sep 18 18:04:14.333025 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.325911    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889" event={"ID":"1592a127-7f26-4b3f-a792-4554cc41aba4","Type":"ContainerDied","Data":"e8bf4f77fe09cb7513e2816a08667ff0efe1e21a398e5cd46e4dec1c3f4a52f9"}
Sep 18 18:04:14.333025 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.325957    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889" event={"ID":"1592a127-7f26-4b3f-a792-4554cc41aba4","Type":"ContainerDied","Data":"33d585c7207991ebac9865e21760eb54cb848c9efc1d1e81f075e1486e90aa0b"}
Sep 18 18:04:14.333025 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.325984    2568 scope.go:117] "RemoveContainer" containerID="e8bf4f77fe09cb7513e2816a08667ff0efe1e21a398e5cd46e4dec1c3f4a52f9"
Sep 18 18:04:14.333025 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.326131    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"
Sep 18 18:04:14.333836 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.333806    2568 generic.go:356] "Generic (PLEG): container finished" podID="7921d9ed-b5d1-4442-a278-ba77f3040f00" containerID="4827e58d81c9d4977459a2849aeb8d70ff7a5401065f8788ab7a2a5eadc78512" exitCode=0
Sep 18 18:04:14.333962 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.333859    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg" event={"ID":"7921d9ed-b5d1-4442-a278-ba77f3040f00","Type":"ContainerDied","Data":"4827e58d81c9d4977459a2849aeb8d70ff7a5401065f8788ab7a2a5eadc78512"}
Sep 18 18:04:14.333962 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.333903    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg" event={"ID":"7921d9ed-b5d1-4442-a278-ba77f3040f00","Type":"ContainerDied","Data":"26a52768457edb97048e0265016987ae3baeb1362e6a4b32326d23d61dc65d57"}
Sep 18 18:04:14.334080 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.333968    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"
Sep 18 18:04:14.342415 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.341506    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1592a127-7f26-4b3f-a792-4554cc41aba4-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:04:14.342415 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.341529    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1592a127-7f26-4b3f-a792-4554cc41aba4-metrics-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:04:14.342415 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.341542    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-272kb\" (UniqueName: \"kubernetes.io/projected/1592a127-7f26-4b3f-a792-4554cc41aba4-kube-api-access-272kb\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:04:14.351042 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.351023    2568 scope.go:117] "RemoveContainer" containerID="e8bf4f77fe09cb7513e2816a08667ff0efe1e21a398e5cd46e4dec1c3f4a52f9"
Sep 18 18:04:14.351356 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:04:14.351334    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8bf4f77fe09cb7513e2816a08667ff0efe1e21a398e5cd46e4dec1c3f4a52f9\": container with ID starting with e8bf4f77fe09cb7513e2816a08667ff0efe1e21a398e5cd46e4dec1c3f4a52f9 not found: ID does not exist" containerID="e8bf4f77fe09cb7513e2816a08667ff0efe1e21a398e5cd46e4dec1c3f4a52f9"
Sep 18 18:04:14.351423 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.351362    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8bf4f77fe09cb7513e2816a08667ff0efe1e21a398e5cd46e4dec1c3f4a52f9"} err="failed to get container status \"e8bf4f77fe09cb7513e2816a08667ff0efe1e21a398e5cd46e4dec1c3f4a52f9\": rpc error: code = NotFound desc = could not find container \"e8bf4f77fe09cb7513e2816a08667ff0efe1e21a398e5cd46e4dec1c3f4a52f9\": container with ID starting with e8bf4f77fe09cb7513e2816a08667ff0efe1e21a398e5cd46e4dec1c3f4a52f9 not found: ID does not exist"
Sep 18 18:04:14.351423 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.351378    2568 scope.go:117] "RemoveContainer" containerID="4827e58d81c9d4977459a2849aeb8d70ff7a5401065f8788ab7a2a5eadc78512"
Sep 18 18:04:14.359729 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.359709    2568 scope.go:117] "RemoveContainer" containerID="4827e58d81c9d4977459a2849aeb8d70ff7a5401065f8788ab7a2a5eadc78512"
Sep 18 18:04:14.360048 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:04:14.360017    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4827e58d81c9d4977459a2849aeb8d70ff7a5401065f8788ab7a2a5eadc78512\": container with ID starting with 4827e58d81c9d4977459a2849aeb8d70ff7a5401065f8788ab7a2a5eadc78512 not found: ID does not exist" containerID="4827e58d81c9d4977459a2849aeb8d70ff7a5401065f8788ab7a2a5eadc78512"
Sep 18 18:04:14.360205 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.360055    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4827e58d81c9d4977459a2849aeb8d70ff7a5401065f8788ab7a2a5eadc78512"} err="failed to get container status \"4827e58d81c9d4977459a2849aeb8d70ff7a5401065f8788ab7a2a5eadc78512\": rpc error: code = NotFound desc = could not find container \"4827e58d81c9d4977459a2849aeb8d70ff7a5401065f8788ab7a2a5eadc78512\": container with ID starting with 4827e58d81c9d4977459a2849aeb8d70ff7a5401065f8788ab7a2a5eadc78512 not found: ID does not exist"
Sep 18 18:04:14.397512 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.397438    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"]
Sep 18 18:04:14.414137 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.414108    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-63da2c01-6876fc7865-sv889"]
Sep 18 18:04:14.442874 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.442850    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czwkl\" (UniqueName: \"kubernetes.io/projected/7921d9ed-b5d1-4442-a278-ba77f3040f00-kube-api-access-czwkl\") pod \"7921d9ed-b5d1-4442-a278-ba77f3040f00\" (UID: \"7921d9ed-b5d1-4442-a278-ba77f3040f00\") "
Sep 18 18:04:14.442989 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.442952    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7921d9ed-b5d1-4442-a278-ba77f3040f00-plugins-config-volume\") pod \"7921d9ed-b5d1-4442-a278-ba77f3040f00\" (UID: \"7921d9ed-b5d1-4442-a278-ba77f3040f00\") "
Sep 18 18:04:14.443279 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.443252    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7921d9ed-b5d1-4442-a278-ba77f3040f00-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7921d9ed-b5d1-4442-a278-ba77f3040f00" (UID: "7921d9ed-b5d1-4442-a278-ba77f3040f00"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:04:14.444873 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.444856    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7921d9ed-b5d1-4442-a278-ba77f3040f00-kube-api-access-czwkl" (OuterVolumeSpecName: "kube-api-access-czwkl") pod "7921d9ed-b5d1-4442-a278-ba77f3040f00" (UID: "7921d9ed-b5d1-4442-a278-ba77f3040f00"). InnerVolumeSpecName "kube-api-access-czwkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:04:14.544361 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.544329    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czwkl\" (UniqueName: \"kubernetes.io/projected/7921d9ed-b5d1-4442-a278-ba77f3040f00-kube-api-access-czwkl\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:04:14.544361 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.544361    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7921d9ed-b5d1-4442-a278-ba77f3040f00-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:04:14.661900 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.661869    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"]
Sep 18 18:04:14.665496 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:14.665468    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-63da2c01-7fdbdf6654-r2ztg"]
Sep 18 18:04:15.547791 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.547761    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"]
Sep 18 18:04:15.548208 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.548194    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1592a127-7f26-4b3f-a792-4554cc41aba4" containerName="maas-api"
Sep 18 18:04:15.548208 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.548209    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1592a127-7f26-4b3f-a792-4554cc41aba4" containerName="maas-api"
Sep 18 18:04:15.548302 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.548222    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7921d9ed-b5d1-4442-a278-ba77f3040f00" containerName="payload-pre-processing"
Sep 18 18:04:15.548302 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.548230    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7921d9ed-b5d1-4442-a278-ba77f3040f00" containerName="payload-pre-processing"
Sep 18 18:04:15.548366 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.548307    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7921d9ed-b5d1-4442-a278-ba77f3040f00" containerName="payload-pre-processing"
Sep 18 18:04:15.548366 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.548314    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1592a127-7f26-4b3f-a792-4554cc41aba4" containerName="maas-api"
Sep 18 18:04:15.551000 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.550982    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.554272 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.554253    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-9f929078-gw-openshift-default-dockercfg-pt9hg\""
Sep 18 18:04:15.571154 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.571131    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"]
Sep 18 18:04:15.653472 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.653440    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-workload-certs\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.653472 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.653472    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-podinfo\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.653737 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.653513    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-credential-socket\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.653737 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.653614    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-workload-socket\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.653737 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.653650    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fhmk\" (UniqueName: \"kubernetes.io/projected/d3801bd8-075a-4185-866b-bf5d7a6083b8-kube-api-access-8fhmk\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.653737 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.653732    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-envoy\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.653959 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.653764    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-data\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.653959 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.653819    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-token\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.653959 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.653850    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3801bd8-075a-4185-866b-bf5d7a6083b8-istiod-ca-cert\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.754576 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.754542    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-credential-socket\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.754770 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.754594    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-workload-socket\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.754770 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.754621    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fhmk\" (UniqueName: \"kubernetes.io/projected/d3801bd8-075a-4185-866b-bf5d7a6083b8-kube-api-access-8fhmk\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.754770 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.754657    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-envoy\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.754962 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.754812    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-data\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.755057 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.755016    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-credential-socket\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.755200 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.755063    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-workload-socket\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.755200 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.755170    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-data\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.755300 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.755244    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-token\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.755300 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.755275    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3801bd8-075a-4185-866b-bf5d7a6083b8-istiod-ca-cert\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.755399 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.755376    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-workload-certs\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.755459 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.755402    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-podinfo\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.755754 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.755723    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-workload-certs\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.756072 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.756045    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3801bd8-075a-4185-866b-bf5d7a6083b8-istiod-ca-cert\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.757309 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.757282    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-envoy\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.758246 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.758220    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-podinfo\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.772911 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.772889    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-token\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.774559 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.774539    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fhmk\" (UniqueName: \"kubernetes.io/projected/d3801bd8-075a-4185-866b-bf5d7a6083b8-kube-api-access-8fhmk\") pod \"e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") " pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:15.862225 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:15.862131    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:16.046710 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:16.046676    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1592a127-7f26-4b3f-a792-4554cc41aba4" path="/var/lib/kubelet/pods/1592a127-7f26-4b3f-a792-4554cc41aba4/volumes"
Sep 18 18:04:16.047208 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:16.047170    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7921d9ed-b5d1-4442-a278-ba77f3040f00" path="/var/lib/kubelet/pods/7921d9ed-b5d1-4442-a278-ba77f3040f00/volumes"
Sep 18 18:04:16.167597 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:16.167572    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"]
Sep 18 18:04:16.170056 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:04:16.170030    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3801bd8_075a_4185_866b_bf5d7a6083b8.slice/crio-b57f3a8da5be30afed172465e678c2b50a24e2bef274a50f189067ba78ac02a6 WatchSource:0}: Error finding container b57f3a8da5be30afed172465e678c2b50a24e2bef274a50f189067ba78ac02a6: Status 404 returned error can't find the container with id b57f3a8da5be30afed172465e678c2b50a24e2bef274a50f189067ba78ac02a6
Sep 18 18:04:16.172445 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:16.172413    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:04:16.172516 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:16.172475    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:04:16.172516 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:16.172504    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:04:16.350654 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:16.350618    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6" event={"ID":"d3801bd8-075a-4185-866b-bf5d7a6083b8","Type":"ContainerStarted","Data":"9263c44444d6d438a951efe414bf5598c78c71ee4ee7df06017e3d0b4794f609"}
Sep 18 18:04:16.350837 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:16.350680    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6" event={"ID":"d3801bd8-075a-4185-866b-bf5d7a6083b8","Type":"ContainerStarted","Data":"b57f3a8da5be30afed172465e678c2b50a24e2bef274a50f189067ba78ac02a6"}
Sep 18 18:04:16.389840 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:16.389777    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6" podStartSLOduration=1.389756662 podStartE2EDuration="1.389756662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:04:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:04:16.387699334 +0000 UTC m=+860.465503234" watchObservedRunningTime="2026-09-18 18:04:16.389756662 +0000 UTC m=+860.467560562"
Sep 18 18:04:16.862647 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:16.862611    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:17.868423 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:17.868389    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:18.363703 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:18.363669    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:18.364590 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:18.364565    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:20.775363 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:20.775327    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:04:23.671413 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:23.671380    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"]
Sep 18 18:04:23.674761 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:23.674741    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:23.678859 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:23.678841    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-9f929078-serving-cert\""
Sep 18 18:04:23.688164 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:23.688140    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"]
Sep 18 18:04:23.730070 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:23.730045    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bvmx\" (UniqueName: \"kubernetes.io/projected/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-kube-api-access-7bvmx\") pod \"maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:23.730214 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:23.730090    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-metrics-certs\") pod \"maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:23.730268 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:23.730229    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-openshift-service-ca\") pod \"maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:23.730268 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:23.730261    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-maas-api-tls\") pod \"maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:23.831397 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:23.831349    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bvmx\" (UniqueName: \"kubernetes.io/projected/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-kube-api-access-7bvmx\") pod \"maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:23.831561 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:23.831414    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-metrics-certs\") pod \"maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:23.831561 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:23.831492    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-openshift-service-ca\") pod \"maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:23.831561 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:23.831521    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-maas-api-tls\") pod \"maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:23.831669 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:04:23.831623    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-serving-cert: secret "maas-api-e2e-migrate-9f929078-serving-cert" not found
Sep 18 18:04:23.831709 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:04:23.831700    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-maas-api-tls podName:a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50 nodeName:}" failed. No retries permitted until 2026-09-18 18:04:24.331680273 +0000 UTC m=+868.409484156 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-maas-api-tls") pod "maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd" (UID: "a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50") : secret "maas-api-e2e-migrate-9f929078-serving-cert" not found
Sep 18 18:04:23.832117 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:23.832088    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-openshift-service-ca\") pod \"maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:23.833854 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:23.833837    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-metrics-certs\") pod \"maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:23.857054 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:23.857033    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bvmx\" (UniqueName: \"kubernetes.io/projected/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-kube-api-access-7bvmx\") pod \"maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:24.336413 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:24.336374    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-maas-api-tls\") pod \"maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:24.338756 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:24.338727    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-maas-api-tls\") pod \"maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:24.589438 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:24.589360    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:24.890543 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:24.890469    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"]
Sep 18 18:04:24.894166 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:04:24.894136    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4c99aa0_4a77_49d1_b09d_1dd7a4a23f50.slice/crio-ae604df470134d41c5f631e60d7cb22445b3809e7ec76872b7fbcc335b7d13a9 WatchSource:0}: Error finding container ae604df470134d41c5f631e60d7cb22445b3809e7ec76872b7fbcc335b7d13a9: Status 404 returned error can't find the container with id ae604df470134d41c5f631e60d7cb22445b3809e7ec76872b7fbcc335b7d13a9
Sep 18 18:04:25.406777 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:25.406751    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd" event={"ID":"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50","Type":"ContainerStarted","Data":"ae604df470134d41c5f631e60d7cb22445b3809e7ec76872b7fbcc335b7d13a9"}
Sep 18 18:04:26.413592 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:26.413554    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd" event={"ID":"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50","Type":"ContainerStarted","Data":"18542f68efa0b4a035f2a8997fecf7b0398a9a5f5f3d602965f0c78deb2cd84f"}
Sep 18 18:04:26.413992 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:26.413695    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:26.466014 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:26.465961    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd" podStartSLOduration=2.96323886 podStartE2EDuration="3.465945487s" totalImagesPullingTime="502.706627ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:04:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:04:26.464349646 +0000 UTC m=+870.542153546" watchObservedRunningTime="2026-09-18 18:04:26.465945487 +0000 UTC m=+870.543749389"
Sep 18 18:04:32.424076 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:32.424043    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:04:40.153768 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:40.153726    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:04:40.502911 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:40.502835    2568 generic.go:356] "Generic (PLEG): container finished" podID="d3801bd8-075a-4185-866b-bf5d7a6083b8" containerID="9263c44444d6d438a951efe414bf5598c78c71ee4ee7df06017e3d0b4794f609" exitCode=0
Sep 18 18:04:40.503072 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:40.502911    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6" event={"ID":"d3801bd8-075a-4185-866b-bf5d7a6083b8","Type":"ContainerDied","Data":"9263c44444d6d438a951efe414bf5598c78c71ee4ee7df06017e3d0b4794f609"}
Sep 18 18:04:40.503343 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:40.503328    2568 scope.go:117] "RemoveContainer" containerID="9263c44444d6d438a951efe414bf5598c78c71ee4ee7df06017e3d0b4794f609"
Sep 18 18:04:40.504532 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:40.504503    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:04:40.504609 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:40.504573    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:04:40.504609 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:40.504605    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:04:41.510552 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:41.510509    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6" event={"ID":"d3801bd8-075a-4185-866b-bf5d7a6083b8","Type":"ContainerStarted","Data":"e161641cec55776e756e9dc05ea9c23318cd53024733ca3f72bc9a423f46e41f"}
Sep 18 18:04:41.863668 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:41.863566    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:41.863668 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:41.863621    2568 patch_prober.go:28] interesting pod/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 18 18:04:41.863923 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:41.863683    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6" podUID="d3801bd8-075a-4185-866b-bf5d7a6083b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 18 18:04:42.863352 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:42.863312    2568 patch_prober.go:28] interesting pod/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 18 18:04:42.863835 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:42.863386    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6" podUID="d3801bd8-075a-4185-866b-bf5d7a6083b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 18 18:04:43.863273 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:43.863236    2568 patch_prober.go:28] interesting pod/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 18 18:04:43.863579 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:43.863546    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6" podUID="d3801bd8-075a-4185-866b-bf5d7a6083b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 18 18:04:44.863637 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:44.863590    2568 patch_prober.go:28] interesting pod/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 18 18:04:44.864142 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:44.863698    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6" podUID="d3801bd8-075a-4185-866b-bf5d7a6083b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 18 18:04:45.863837 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:45.863797    2568 patch_prober.go:28] interesting pod/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 18 18:04:45.864349 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:45.863874    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6" podUID="d3801bd8-075a-4185-866b-bf5d7a6083b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 18 18:04:46.867465 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:46.867434    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:46.867857 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:46.867703    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:46.868325 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:46.868310    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:04:55.426647 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.426613    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"]
Sep 18 18:04:55.429254 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.429239    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.432760 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.432735    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-609aba\"/\"kube-root-ca.crt\""
Sep 18 18:04:55.433292 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.433277    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-609aba\"/\"default-dockercfg-4hgmq\""
Sep 18 18:04:55.433383 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.433300    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-609aba\"/\"simulator-w0-609aba-kserve-self-signed-certs\""
Sep 18 18:04:55.434115 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.434096    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-609aba\"/\"openshift-service-ca.crt\""
Sep 18 18:04:55.452760 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.452736    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-tls-certs\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.452890 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.452782    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-tmp-dir\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.452890 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.452808    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-home\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.452890 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.452853    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-model-cache\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.452890 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.452886    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-dshm\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.453074 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.452953    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l5bh\" (UniqueName: \"kubernetes.io/projected/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-kube-api-access-5l5bh\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.497144 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.497121    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"]
Sep 18 18:04:55.553501 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.553480    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l5bh\" (UniqueName: \"kubernetes.io/projected/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-kube-api-access-5l5bh\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.553616 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.553567    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-tls-certs\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.553616 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.553589    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-tmp-dir\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.553616 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.553604    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-home\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.553737 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.553627    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-model-cache\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.553737 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.553657    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-dshm\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.553978 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.553958    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-home\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.554072 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.554054    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-model-cache\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.554108 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.554070    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-tmp-dir\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.555847 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.555831    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-dshm\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.556128 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.556111    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-tls-certs\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.579108 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.579060    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l5bh\" (UniqueName: \"kubernetes.io/projected/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-kube-api-access-5l5bh\") pod \"simulator-w0-609aba-kserve-68678954c4-knfzv\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") " pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:55.738520 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:55.738439    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:04:56.048657 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:56.048627    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"]
Sep 18 18:04:56.049553 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:04:56.049531    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55e9632e_d0c7_456d_a2f8_a3d998e8cd29.slice/crio-69f305cc0825d831be67f6e79bdc6cd2d6e5e791231dbc60dc9f13bbbe9bcd0d WatchSource:0}: Error finding container 69f305cc0825d831be67f6e79bdc6cd2d6e5e791231dbc60dc9f13bbbe9bcd0d: Status 404 returned error can't find the container with id 69f305cc0825d831be67f6e79bdc6cd2d6e5e791231dbc60dc9f13bbbe9bcd0d
Sep 18 18:04:56.597111 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:56.597077    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv" event={"ID":"55e9632e-d0c7-456d-a2f8-a3d998e8cd29","Type":"ContainerStarted","Data":"19011cc33a3095e322a0eb3f4366e5547f1956ebd9242a32775c07b3ef379e2a"}
Sep 18 18:04:56.597596 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:56.597120    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv" event={"ID":"55e9632e-d0c7-456d-a2f8-a3d998e8cd29","Type":"ContainerStarted","Data":"69f305cc0825d831be67f6e79bdc6cd2d6e5e791231dbc60dc9f13bbbe9bcd0d"}
Sep 18 18:04:56.624634 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:56.624587    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv" podStartSLOduration=1.624571977 podStartE2EDuration="1.624571977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:04:56.62289142 +0000 UTC m=+900.700695319" watchObservedRunningTime="2026-09-18 18:04:56.624571977 +0000 UTC m=+900.702375928"
Sep 18 18:04:58.017235 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:58.017213    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 18:04:58.019610 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:04:58.019590    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 18:05:00.048170 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.048132    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"]
Sep 18 18:05:00.048571 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.048344    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd" podUID="a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50" containerName="maas-api" containerID="cri-o://18542f68efa0b4a035f2a8997fecf7b0398a9a5f5f3d602965f0c78deb2cd84f" gracePeriod=30
Sep 18 18:05:00.309705 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.309634    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:05:00.399382 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.399350    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-openshift-service-ca\") pod \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") "
Sep 18 18:05:00.399553 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.399392    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-metrics-certs\") pod \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") "
Sep 18 18:05:00.399603 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.399572    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-maas-api-tls\") pod \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") "
Sep 18 18:05:00.399698 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.399681    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bvmx\" (UniqueName: \"kubernetes.io/projected/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-kube-api-access-7bvmx\") pod \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\" (UID: \"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50\") "
Sep 18 18:05:00.399771 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.399691    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50" (UID: "a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:05:00.399973 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.399952    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:00.401597 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.401573    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50" (UID: "a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:05:00.401689 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.401604    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50" (UID: "a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:05:00.401689 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.401619    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-kube-api-access-7bvmx" (OuterVolumeSpecName: "kube-api-access-7bvmx") pod "a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50" (UID: "a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50"). InnerVolumeSpecName "kube-api-access-7bvmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:05:00.500715 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.500676    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bvmx\" (UniqueName: \"kubernetes.io/projected/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-kube-api-access-7bvmx\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:00.500715 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.500718    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-metrics-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:00.500912 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.500733    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:00.622333 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.622245    2568 generic.go:356] "Generic (PLEG): container finished" podID="a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50" containerID="18542f68efa0b4a035f2a8997fecf7b0398a9a5f5f3d602965f0c78deb2cd84f" exitCode=0
Sep 18 18:05:00.622333 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.622293    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"
Sep 18 18:05:00.622522 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.622332    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd" event={"ID":"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50","Type":"ContainerDied","Data":"18542f68efa0b4a035f2a8997fecf7b0398a9a5f5f3d602965f0c78deb2cd84f"}
Sep 18 18:05:00.622522 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.622393    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd" event={"ID":"a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50","Type":"ContainerDied","Data":"ae604df470134d41c5f631e60d7cb22445b3809e7ec76872b7fbcc335b7d13a9"}
Sep 18 18:05:00.622522 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.622423    2568 scope.go:117] "RemoveContainer" containerID="18542f68efa0b4a035f2a8997fecf7b0398a9a5f5f3d602965f0c78deb2cd84f"
Sep 18 18:05:00.632071 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.632051    2568 scope.go:117] "RemoveContainer" containerID="18542f68efa0b4a035f2a8997fecf7b0398a9a5f5f3d602965f0c78deb2cd84f"
Sep 18 18:05:00.632398 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:05:00.632372    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18542f68efa0b4a035f2a8997fecf7b0398a9a5f5f3d602965f0c78deb2cd84f\": container with ID starting with 18542f68efa0b4a035f2a8997fecf7b0398a9a5f5f3d602965f0c78deb2cd84f not found: ID does not exist" containerID="18542f68efa0b4a035f2a8997fecf7b0398a9a5f5f3d602965f0c78deb2cd84f"
Sep 18 18:05:00.632488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.632412    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18542f68efa0b4a035f2a8997fecf7b0398a9a5f5f3d602965f0c78deb2cd84f"} err="failed to get container status \"18542f68efa0b4a035f2a8997fecf7b0398a9a5f5f3d602965f0c78deb2cd84f\": rpc error: code = NotFound desc = could not find container \"18542f68efa0b4a035f2a8997fecf7b0398a9a5f5f3d602965f0c78deb2cd84f\": container with ID starting with 18542f68efa0b4a035f2a8997fecf7b0398a9a5f5f3d602965f0c78deb2cd84f not found: ID does not exist"
Sep 18 18:05:00.660289 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.660259    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"]
Sep 18 18:05:00.670625 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.670602    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9f929078-dbf694fb4-hxvbd"]
Sep 18 18:05:00.748673 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.748642    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"]
Sep 18 18:05:00.748959 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:00.748915    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6" podUID="d3801bd8-075a-4185-866b-bf5d7a6083b8" containerName="istio-proxy" containerID="cri-o://e161641cec55776e756e9dc05ea9c23318cd53024733ca3f72bc9a423f46e41f" gracePeriod=30
Sep 18 18:05:01.870751 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:01.870711    2568 patch_prober.go:28] interesting pod/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 18:05:01.871131 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:01.870771    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6" podUID="d3801bd8-075a-4185-866b-bf5d7a6083b8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:05:02.046009 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:02.045974    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50" path="/var/lib/kubelet/pods/a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50/volumes"
Sep 18 18:05:05.738752 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:05.738709    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:05:05.738752 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:05.738760    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:05:05.751530 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:05.751496    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:05:06.006980 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.006951    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:05:06.018685 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.018658    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:05:06.055158 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055127    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-workload-certs\") pod \"d3801bd8-075a-4185-866b-bf5d7a6083b8\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") "
Sep 18 18:05:06.055360 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055204    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-token\") pod \"d3801bd8-075a-4185-866b-bf5d7a6083b8\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") "
Sep 18 18:05:06.055360 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055234    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-credential-socket\") pod \"d3801bd8-075a-4185-866b-bf5d7a6083b8\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") "
Sep 18 18:05:06.055360 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055258    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-data\") pod \"d3801bd8-075a-4185-866b-bf5d7a6083b8\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") "
Sep 18 18:05:06.055360 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055286    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3801bd8-075a-4185-866b-bf5d7a6083b8-istiod-ca-cert\") pod \"d3801bd8-075a-4185-866b-bf5d7a6083b8\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") "
Sep 18 18:05:06.055360 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055335    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-envoy\") pod \"d3801bd8-075a-4185-866b-bf5d7a6083b8\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") "
Sep 18 18:05:06.055628 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055389    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-workload-socket\") pod \"d3801bd8-075a-4185-866b-bf5d7a6083b8\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") "
Sep 18 18:05:06.055628 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055430    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-podinfo\") pod \"d3801bd8-075a-4185-866b-bf5d7a6083b8\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") "
Sep 18 18:05:06.055628 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055453    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d3801bd8-075a-4185-866b-bf5d7a6083b8" (UID: "d3801bd8-075a-4185-866b-bf5d7a6083b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:05:06.055628 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055473    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fhmk\" (UniqueName: \"kubernetes.io/projected/d3801bd8-075a-4185-866b-bf5d7a6083b8-kube-api-access-8fhmk\") pod \"d3801bd8-075a-4185-866b-bf5d7a6083b8\" (UID: \"d3801bd8-075a-4185-866b-bf5d7a6083b8\") "
Sep 18 18:05:06.055628 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055561    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d3801bd8-075a-4185-866b-bf5d7a6083b8" (UID: "d3801bd8-075a-4185-866b-bf5d7a6083b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:05:06.055628 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055579    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "d3801bd8-075a-4185-866b-bf5d7a6083b8" (UID: "d3801bd8-075a-4185-866b-bf5d7a6083b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:05:06.055938 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055738    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d3801bd8-075a-4185-866b-bf5d7a6083b8" (UID: "d3801bd8-075a-4185-866b-bf5d7a6083b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:05:06.055938 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055919    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:06.056050 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055940    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:06.056050 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055955    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:06.056050 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.055967    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:06.056236 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.056045    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3801bd8-075a-4185-866b-bf5d7a6083b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d3801bd8-075a-4185-866b-bf5d7a6083b8" (UID: "d3801bd8-075a-4185-866b-bf5d7a6083b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:05:06.058472 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.058432    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d3801bd8-075a-4185-866b-bf5d7a6083b8" (UID: "d3801bd8-075a-4185-866b-bf5d7a6083b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:05:06.059536 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.059500    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d3801bd8-075a-4185-866b-bf5d7a6083b8" (UID: "d3801bd8-075a-4185-866b-bf5d7a6083b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:05:06.059647 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.059510    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "d3801bd8-075a-4185-866b-bf5d7a6083b8" (UID: "d3801bd8-075a-4185-866b-bf5d7a6083b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:05:06.059647 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.059585    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3801bd8-075a-4185-866b-bf5d7a6083b8-kube-api-access-8fhmk" (OuterVolumeSpecName: "kube-api-access-8fhmk") pod "d3801bd8-075a-4185-866b-bf5d7a6083b8" (UID: "d3801bd8-075a-4185-866b-bf5d7a6083b8"). InnerVolumeSpecName "kube-api-access-8fhmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:05:06.157421 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.157386    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:06.157421 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.157421    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3801bd8-075a-4185-866b-bf5d7a6083b8-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:06.157421 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.157438    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:06.157738 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.157451    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3801bd8-075a-4185-866b-bf5d7a6083b8-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:06.157738 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.157460    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fhmk\" (UniqueName: \"kubernetes.io/projected/d3801bd8-075a-4185-866b-bf5d7a6083b8-kube-api-access-8fhmk\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:06.659951 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.659913    2568 generic.go:356] "Generic (PLEG): container finished" podID="d3801bd8-075a-4185-866b-bf5d7a6083b8" containerID="e161641cec55776e756e9dc05ea9c23318cd53024733ca3f72bc9a423f46e41f" exitCode=0
Sep 18 18:05:06.660157 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.659980    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"
Sep 18 18:05:06.660157 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.660090    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6" event={"ID":"d3801bd8-075a-4185-866b-bf5d7a6083b8","Type":"ContainerDied","Data":"e161641cec55776e756e9dc05ea9c23318cd53024733ca3f72bc9a423f46e41f"}
Sep 18 18:05:06.660157 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.660139    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6" event={"ID":"d3801bd8-075a-4185-866b-bf5d7a6083b8","Type":"ContainerDied","Data":"b57f3a8da5be30afed172465e678c2b50a24e2bef274a50f189067ba78ac02a6"}
Sep 18 18:05:06.660374 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.660166    2568 scope.go:117] "RemoveContainer" containerID="e161641cec55776e756e9dc05ea9c23318cd53024733ca3f72bc9a423f46e41f"
Sep 18 18:05:06.672458 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.672437    2568 scope.go:117] "RemoveContainer" containerID="9263c44444d6d438a951efe414bf5598c78c71ee4ee7df06017e3d0b4794f609"
Sep 18 18:05:06.675943 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.675918    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:05:06.681881 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.681856    2568 scope.go:117] "RemoveContainer" containerID="e161641cec55776e756e9dc05ea9c23318cd53024733ca3f72bc9a423f46e41f"
Sep 18 18:05:06.682261 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:05:06.682171    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e161641cec55776e756e9dc05ea9c23318cd53024733ca3f72bc9a423f46e41f\": container with ID starting with e161641cec55776e756e9dc05ea9c23318cd53024733ca3f72bc9a423f46e41f not found: ID does not exist" containerID="e161641cec55776e756e9dc05ea9c23318cd53024733ca3f72bc9a423f46e41f"
Sep 18 18:05:06.682261 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.682235    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e161641cec55776e756e9dc05ea9c23318cd53024733ca3f72bc9a423f46e41f"} err="failed to get container status \"e161641cec55776e756e9dc05ea9c23318cd53024733ca3f72bc9a423f46e41f\": rpc error: code = NotFound desc = could not find container \"e161641cec55776e756e9dc05ea9c23318cd53024733ca3f72bc9a423f46e41f\": container with ID starting with e161641cec55776e756e9dc05ea9c23318cd53024733ca3f72bc9a423f46e41f not found: ID does not exist"
Sep 18 18:05:06.682261 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.682253    2568 scope.go:117] "RemoveContainer" containerID="9263c44444d6d438a951efe414bf5598c78c71ee4ee7df06017e3d0b4794f609"
Sep 18 18:05:06.682544 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:05:06.682525    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9263c44444d6d438a951efe414bf5598c78c71ee4ee7df06017e3d0b4794f609\": container with ID starting with 9263c44444d6d438a951efe414bf5598c78c71ee4ee7df06017e3d0b4794f609 not found: ID does not exist" containerID="9263c44444d6d438a951efe414bf5598c78c71ee4ee7df06017e3d0b4794f609"
Sep 18 18:05:06.682609 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.682549    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9263c44444d6d438a951efe414bf5598c78c71ee4ee7df06017e3d0b4794f609"} err="failed to get container status \"9263c44444d6d438a951efe414bf5598c78c71ee4ee7df06017e3d0b4794f609\": rpc error: code = NotFound desc = could not find container \"9263c44444d6d438a951efe414bf5598c78c71ee4ee7df06017e3d0b4794f609\": container with ID starting with 9263c44444d6d438a951efe414bf5598c78c71ee4ee7df06017e3d0b4794f609 not found: ID does not exist"
Sep 18 18:05:06.693454 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.693420    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"]
Sep 18 18:05:06.703368 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:06.703314    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-9f929078-gw-openshift-default-6bf45c9988-skjs6"]
Sep 18 18:05:08.044992 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.044957    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3801bd8-075a-4185-866b-bf5d7a6083b8" path="/var/lib/kubelet/pods/d3801bd8-075a-4185-866b-bf5d7a6083b8/volumes"
Sep 18 18:05:08.430330 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.430292    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"]
Sep 18 18:05:08.430809 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.430791    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3801bd8-075a-4185-866b-bf5d7a6083b8" containerName="istio-proxy"
Sep 18 18:05:08.430906 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.430811    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3801bd8-075a-4185-866b-bf5d7a6083b8" containerName="istio-proxy"
Sep 18 18:05:08.430906 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.430829    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50" containerName="maas-api"
Sep 18 18:05:08.430906 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.430836    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50" containerName="maas-api"
Sep 18 18:05:08.431069 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.430980    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4c99aa0-4a77-49d1-b09d-1dd7a4a23f50" containerName="maas-api"
Sep 18 18:05:08.431069 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.430997    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3801bd8-075a-4185-866b-bf5d7a6083b8" containerName="istio-proxy"
Sep 18 18:05:08.431069 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.431006    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3801bd8-075a-4185-866b-bf5d7a6083b8" containerName="istio-proxy"
Sep 18 18:05:08.433849 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.433826    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:08.440787 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.440766    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-284e39c5-serving-cert\""
Sep 18 18:05:08.458128 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.458101    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"]
Sep 18 18:05:08.481116 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.481074    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc9ea665-6300-4ba5-beb6-8de552fa9960-metrics-certs\") pod \"maas-api-e2e-ait-284e39c5-869c455694-65dl6\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:08.481343 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.481133    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9ea665-6300-4ba5-beb6-8de552fa9960-maas-api-tls\") pod \"maas-api-e2e-ait-284e39c5-869c455694-65dl6\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:08.481343 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.481201    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc9ea665-6300-4ba5-beb6-8de552fa9960-openshift-service-ca\") pod \"maas-api-e2e-ait-284e39c5-869c455694-65dl6\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:08.481343 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.481249    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzzpc\" (UniqueName: \"kubernetes.io/projected/dc9ea665-6300-4ba5-beb6-8de552fa9960-kube-api-access-gzzpc\") pod \"maas-api-e2e-ait-284e39c5-869c455694-65dl6\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:08.582845 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.582804    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc9ea665-6300-4ba5-beb6-8de552fa9960-metrics-certs\") pod \"maas-api-e2e-ait-284e39c5-869c455694-65dl6\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:08.583025 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.582912    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9ea665-6300-4ba5-beb6-8de552fa9960-maas-api-tls\") pod \"maas-api-e2e-ait-284e39c5-869c455694-65dl6\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:08.583025 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.582958    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc9ea665-6300-4ba5-beb6-8de552fa9960-openshift-service-ca\") pod \"maas-api-e2e-ait-284e39c5-869c455694-65dl6\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:08.583025 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.582995    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzzpc\" (UniqueName: \"kubernetes.io/projected/dc9ea665-6300-4ba5-beb6-8de552fa9960-kube-api-access-gzzpc\") pod \"maas-api-e2e-ait-284e39c5-869c455694-65dl6\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:08.583633 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.583608    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc9ea665-6300-4ba5-beb6-8de552fa9960-openshift-service-ca\") pod \"maas-api-e2e-ait-284e39c5-869c455694-65dl6\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:08.585390 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.585367    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc9ea665-6300-4ba5-beb6-8de552fa9960-metrics-certs\") pod \"maas-api-e2e-ait-284e39c5-869c455694-65dl6\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:08.585478 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.585441    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9ea665-6300-4ba5-beb6-8de552fa9960-maas-api-tls\") pod \"maas-api-e2e-ait-284e39c5-869c455694-65dl6\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:08.596050 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.596022    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzzpc\" (UniqueName: \"kubernetes.io/projected/dc9ea665-6300-4ba5-beb6-8de552fa9960-kube-api-access-gzzpc\") pod \"maas-api-e2e-ait-284e39c5-869c455694-65dl6\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:08.746695 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:08.746603    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:09.043040 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:09.043014    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"]
Sep 18 18:05:09.045437 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:05:09.045399    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc9ea665_6300_4ba5_beb6_8de552fa9960.slice/crio-874f1234fc5b91ff6b0deab81a58e6d9f797b6dd170447c0c4461a0312272221 WatchSource:0}: Error finding container 874f1234fc5b91ff6b0deab81a58e6d9f797b6dd170447c0c4461a0312272221: Status 404 returned error can't find the container with id 874f1234fc5b91ff6b0deab81a58e6d9f797b6dd170447c0c4461a0312272221
Sep 18 18:05:09.678131 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:09.678093    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6" event={"ID":"dc9ea665-6300-4ba5-beb6-8de552fa9960","Type":"ContainerStarted","Data":"fe6fe041e451c8fd247fa9e780ff7b1d639e6dead0195af10dafde8732ee1eb4"}
Sep 18 18:05:09.678328 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:09.678208    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:09.678328 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:09.678229    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6" event={"ID":"dc9ea665-6300-4ba5-beb6-8de552fa9960","Type":"ContainerStarted","Data":"874f1234fc5b91ff6b0deab81a58e6d9f797b6dd170447c0c4461a0312272221"}
Sep 18 18:05:09.731064 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:09.730999    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6" podStartSLOduration=1.260897077 podStartE2EDuration="1.730978557s" totalImagesPullingTime="470.08148ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:05:09.730031855 +0000 UTC m=+913.807835755" watchObservedRunningTime="2026-09-18 18:05:09.730978557 +0000 UTC m=+913.808782459"
Sep 18 18:05:15.689466 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:15.689436    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:30.995423 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:30.995375    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:05:40.787126 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.787085    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"]
Sep 18 18:05:40.787612 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.787546    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3801bd8-075a-4185-866b-bf5d7a6083b8" containerName="istio-proxy"
Sep 18 18:05:40.787612 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.787558    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3801bd8-075a-4185-866b-bf5d7a6083b8" containerName="istio-proxy"
Sep 18 18:05:40.789993 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.789968    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.794150 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.794126    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-502939\"/\"kube-root-ca.crt\""
Sep 18 18:05:40.795055 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.795028    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-502939\"/\"default-dockercfg-xhqmb\""
Sep 18 18:05:40.795207 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.795087    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-502939\"/\"test-model-502939-kserve-self-signed-certs\""
Sep 18 18:05:40.795207 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.795151    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-502939\"/\"openshift-service-ca.crt\""
Sep 18 18:05:40.802982 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.802959    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-home\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.803099 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.803006    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-dshm\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.803099 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.803025    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-model-cache\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.803099 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.803042    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-tmp-dir\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.803099 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.803062    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt2bq\" (UniqueName: \"kubernetes.io/projected/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-kube-api-access-rt2bq\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.803342 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.803172    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-tls-certs\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.809916 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.809892    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"]
Sep 18 18:05:40.904607 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.904569    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-home\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.904827 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.904634    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-dshm\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.904827 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.904653    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-model-cache\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.904827 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.904672    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-tmp-dir\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.904827 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.904688    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt2bq\" (UniqueName: \"kubernetes.io/projected/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-kube-api-access-rt2bq\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.904827 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.904747    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-tls-certs\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.905118 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.905035    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-home\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.905118 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.905059    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-model-cache\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.905250 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.905169    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-tmp-dir\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.907115 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.907091    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-dshm\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.907420 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.907400    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-tls-certs\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:40.920341 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:40.920316    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt2bq\" (UniqueName: \"kubernetes.io/projected/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-kube-api-access-rt2bq\") pod \"test-model-502939-kserve-65848f44fc-59wz9\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") " pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:41.100959 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:41.100867    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:41.386668 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:41.386643    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"]
Sep 18 18:05:41.388843 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:05:41.388813    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6da81ff4_ca04_4289_bb42_fe0e5a9828b0.slice/crio-62b2a0b1a36333a941b5c34e32dbab5e54985a1360bf0639b9447308dbd4c15c WatchSource:0}: Error finding container 62b2a0b1a36333a941b5c34e32dbab5e54985a1360bf0639b9447308dbd4c15c: Status 404 returned error can't find the container with id 62b2a0b1a36333a941b5c34e32dbab5e54985a1360bf0639b9447308dbd4c15c
Sep 18 18:05:41.868340 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:41.868301    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9" event={"ID":"6da81ff4-ca04-4289-bb42-fe0e5a9828b0","Type":"ContainerStarted","Data":"354e0dc1576cf1e563cc14ca78d8b89ed2b6513ae1bfc26a220ca2eb3e449312"}
Sep 18 18:05:41.868718 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:41.868354    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9" event={"ID":"6da81ff4-ca04-4289-bb42-fe0e5a9828b0","Type":"ContainerStarted","Data":"62b2a0b1a36333a941b5c34e32dbab5e54985a1360bf0639b9447308dbd4c15c"}
Sep 18 18:05:41.898767 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:41.898715    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9" podStartSLOduration=1.898700412 podStartE2EDuration="1.898700412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:05:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:05:41.897056121 +0000 UTC m=+945.974860042" watchObservedRunningTime="2026-09-18 18:05:41.898700412 +0000 UTC m=+945.976504311"
Sep 18 18:05:47.305133 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.305093    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"]
Sep 18 18:05:47.305557 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.305335    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6" podUID="dc9ea665-6300-4ba5-beb6-8de552fa9960" containerName="maas-api" containerID="cri-o://fe6fe041e451c8fd247fa9e780ff7b1d639e6dead0195af10dafde8732ee1eb4" gracePeriod=30
Sep 18 18:05:47.560562 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.560505    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:47.674114 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.674088    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc9ea665-6300-4ba5-beb6-8de552fa9960-metrics-certs\") pod \"dc9ea665-6300-4ba5-beb6-8de552fa9960\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") "
Sep 18 18:05:47.674354 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.674335    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc9ea665-6300-4ba5-beb6-8de552fa9960-openshift-service-ca\") pod \"dc9ea665-6300-4ba5-beb6-8de552fa9960\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") "
Sep 18 18:05:47.674434 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.674421    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9ea665-6300-4ba5-beb6-8de552fa9960-maas-api-tls\") pod \"dc9ea665-6300-4ba5-beb6-8de552fa9960\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") "
Sep 18 18:05:47.674504 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.674475    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzzpc\" (UniqueName: \"kubernetes.io/projected/dc9ea665-6300-4ba5-beb6-8de552fa9960-kube-api-access-gzzpc\") pod \"dc9ea665-6300-4ba5-beb6-8de552fa9960\" (UID: \"dc9ea665-6300-4ba5-beb6-8de552fa9960\") "
Sep 18 18:05:47.674719 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.674689    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc9ea665-6300-4ba5-beb6-8de552fa9960-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dc9ea665-6300-4ba5-beb6-8de552fa9960" (UID: "dc9ea665-6300-4ba5-beb6-8de552fa9960"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:05:47.674789 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.674777    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc9ea665-6300-4ba5-beb6-8de552fa9960-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:47.676444 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.676419    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc9ea665-6300-4ba5-beb6-8de552fa9960-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "dc9ea665-6300-4ba5-beb6-8de552fa9960" (UID: "dc9ea665-6300-4ba5-beb6-8de552fa9960"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:05:47.676544 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.676485    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc9ea665-6300-4ba5-beb6-8de552fa9960-kube-api-access-gzzpc" (OuterVolumeSpecName: "kube-api-access-gzzpc") pod "dc9ea665-6300-4ba5-beb6-8de552fa9960" (UID: "dc9ea665-6300-4ba5-beb6-8de552fa9960"). InnerVolumeSpecName "kube-api-access-gzzpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:05:47.676585 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.676563    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc9ea665-6300-4ba5-beb6-8de552fa9960-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dc9ea665-6300-4ba5-beb6-8de552fa9960" (UID: "dc9ea665-6300-4ba5-beb6-8de552fa9960"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:05:47.775650 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.775621    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc9ea665-6300-4ba5-beb6-8de552fa9960-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:47.775650 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.775645    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzzpc\" (UniqueName: \"kubernetes.io/projected/dc9ea665-6300-4ba5-beb6-8de552fa9960-kube-api-access-gzzpc\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:47.775650 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.775657    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc9ea665-6300-4ba5-beb6-8de552fa9960-metrics-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:05:47.905481 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.905395    2568 generic.go:356] "Generic (PLEG): container finished" podID="dc9ea665-6300-4ba5-beb6-8de552fa9960" containerID="fe6fe041e451c8fd247fa9e780ff7b1d639e6dead0195af10dafde8732ee1eb4" exitCode=0
Sep 18 18:05:47.905481 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.905441    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"
Sep 18 18:05:47.905481 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.905468    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6" event={"ID":"dc9ea665-6300-4ba5-beb6-8de552fa9960","Type":"ContainerDied","Data":"fe6fe041e451c8fd247fa9e780ff7b1d639e6dead0195af10dafde8732ee1eb4"}
Sep 18 18:05:47.905753 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.905522    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6" event={"ID":"dc9ea665-6300-4ba5-beb6-8de552fa9960","Type":"ContainerDied","Data":"874f1234fc5b91ff6b0deab81a58e6d9f797b6dd170447c0c4461a0312272221"}
Sep 18 18:05:47.905753 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.905546    2568 scope.go:117] "RemoveContainer" containerID="fe6fe041e451c8fd247fa9e780ff7b1d639e6dead0195af10dafde8732ee1eb4"
Sep 18 18:05:47.915246 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.915229    2568 scope.go:117] "RemoveContainer" containerID="fe6fe041e451c8fd247fa9e780ff7b1d639e6dead0195af10dafde8732ee1eb4"
Sep 18 18:05:47.915530 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:05:47.915506    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe6fe041e451c8fd247fa9e780ff7b1d639e6dead0195af10dafde8732ee1eb4\": container with ID starting with fe6fe041e451c8fd247fa9e780ff7b1d639e6dead0195af10dafde8732ee1eb4 not found: ID does not exist" containerID="fe6fe041e451c8fd247fa9e780ff7b1d639e6dead0195af10dafde8732ee1eb4"
Sep 18 18:05:47.915628 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.915535    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe6fe041e451c8fd247fa9e780ff7b1d639e6dead0195af10dafde8732ee1eb4"} err="failed to get container status \"fe6fe041e451c8fd247fa9e780ff7b1d639e6dead0195af10dafde8732ee1eb4\": rpc error: code = NotFound desc = could not find container \"fe6fe041e451c8fd247fa9e780ff7b1d639e6dead0195af10dafde8732ee1eb4\": container with ID starting with fe6fe041e451c8fd247fa9e780ff7b1d639e6dead0195af10dafde8732ee1eb4 not found: ID does not exist"
Sep 18 18:05:47.953613 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.953587    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"]
Sep 18 18:05:47.968112 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:47.968087    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-284e39c5-869c455694-65dl6"]
Sep 18 18:05:48.045963 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:48.045936    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc9ea665-6300-4ba5-beb6-8de552fa9960" path="/var/lib/kubelet/pods/dc9ea665-6300-4ba5-beb6-8de552fa9960/volumes"
Sep 18 18:05:51.101279 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:51.101244    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:51.101667 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:51.101291    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:51.114151 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:51.114124    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:51.939459 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:51.939433    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:05:56.175153 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.175119    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"]
Sep 18 18:05:56.175825 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.175809    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc9ea665-6300-4ba5-beb6-8de552fa9960" containerName="maas-api"
Sep 18 18:05:56.175873 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.175829    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc9ea665-6300-4ba5-beb6-8de552fa9960" containerName="maas-api"
Sep 18 18:05:56.175977 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.175966    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc9ea665-6300-4ba5-beb6-8de552fa9960" containerName="maas-api"
Sep 18 18:05:56.179267 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.179248    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:05:56.183614 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.183597    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-31e0da22-serving-cert\""
Sep 18 18:05:56.260961 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.260932    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-maas-api-tls\") pod \"maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:05:56.261124 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.260996    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-metrics-certs\") pod \"maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:05:56.261124 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.261033    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j68f\" (UniqueName: \"kubernetes.io/projected/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-kube-api-access-2j68f\") pod \"maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:05:56.261124 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.261064    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-openshift-service-ca\") pod \"maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:05:56.290525 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.290496    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"]
Sep 18 18:05:56.362112 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.362081    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j68f\" (UniqueName: \"kubernetes.io/projected/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-kube-api-access-2j68f\") pod \"maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:05:56.362112 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.362120    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-openshift-service-ca\") pod \"maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:05:56.362405 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.362277    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-maas-api-tls\") pod \"maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:05:56.362405 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.362355    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-metrics-certs\") pod \"maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:05:56.363003 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.362977    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-openshift-service-ca\") pod \"maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:05:56.364980 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.364959    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-maas-api-tls\") pod \"maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:05:56.365133 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.365113    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-metrics-certs\") pod \"maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:05:56.383263 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.383241    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j68f\" (UniqueName: \"kubernetes.io/projected/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-kube-api-access-2j68f\") pod \"maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:05:56.490437 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.490350    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:05:56.771326 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.771256    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"]
Sep 18 18:05:56.773812 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:05:56.773783    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b27d75b_fd05_44f7_87c5_a5c42aa8fbc6.slice/crio-cb2406158e9938d595f97f75fb44bb62813039b0ebb99beb17d689e860cb1b2d WatchSource:0}: Error finding container cb2406158e9938d595f97f75fb44bb62813039b0ebb99beb17d689e860cb1b2d: Status 404 returned error can't find the container with id cb2406158e9938d595f97f75fb44bb62813039b0ebb99beb17d689e860cb1b2d
Sep 18 18:05:56.955405 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:56.955372    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5" event={"ID":"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6","Type":"ContainerStarted","Data":"cb2406158e9938d595f97f75fb44bb62813039b0ebb99beb17d689e860cb1b2d"}
Sep 18 18:05:57.965122 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:57.965089    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5" event={"ID":"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6","Type":"ContainerStarted","Data":"9217056368f88a1eca6464fd44f81eb1f1b7e9989c62738162386122fa8287bf"}
Sep 18 18:05:57.965543 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:57.965256    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:05:57.997851 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:57.997804    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5" podStartSLOduration=1.336761057 podStartE2EDuration="1.997789214s" totalImagesPullingTime="661.028157ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:05:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:05:57.994760377 +0000 UTC m=+962.072564276" watchObservedRunningTime="2026-09-18 18:05:57.997789214 +0000 UTC m=+962.075593112"
Sep 18 18:05:59.341744 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:05:59.341715    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:06:03.976736 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:03.976700    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:06:08.780015 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:08.779984    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"]
Sep 18 18:06:08.780433 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:08.780241    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr" podUID="be0f070a-70fd-4268-a019-a45f147315e8" containerName="payload-processing" containerID="cri-o://afa1b06b50987927f3dda4be8f56702ab79478416f6eca43dce8082448252470" gracePeriod=30
Sep 18 18:06:08.786053 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:08.786025    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"]
Sep 18 18:06:08.786299 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:08.786255    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8" podUID="947b2e8f-8ba2-4912-b0f2-f59b76e3f62e" containerName="payload-pre-processing" containerID="cri-o://bcc1dc0fc21b0b229c096f0702c90ccd5886101d40f9224c8e144a07e3e8b102" gracePeriod=30
Sep 18 18:06:09.030087 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.029992    2568 generic.go:356] "Generic (PLEG): container finished" podID="be0f070a-70fd-4268-a019-a45f147315e8" containerID="afa1b06b50987927f3dda4be8f56702ab79478416f6eca43dce8082448252470" exitCode=0
Sep 18 18:06:09.030309 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.030079    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr" event={"ID":"be0f070a-70fd-4268-a019-a45f147315e8","Type":"ContainerDied","Data":"afa1b06b50987927f3dda4be8f56702ab79478416f6eca43dce8082448252470"}
Sep 18 18:06:09.031962 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.031939    2568 generic.go:356] "Generic (PLEG): container finished" podID="947b2e8f-8ba2-4912-b0f2-f59b76e3f62e" containerID="bcc1dc0fc21b0b229c096f0702c90ccd5886101d40f9224c8e144a07e3e8b102" exitCode=0
Sep 18 18:06:09.032063 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.031964    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8" event={"ID":"947b2e8f-8ba2-4912-b0f2-f59b76e3f62e","Type":"ContainerDied","Data":"bcc1dc0fc21b0b229c096f0702c90ccd5886101d40f9224c8e144a07e3e8b102"}
Sep 18 18:06:09.072011 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.071987    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"
Sep 18 18:06:09.075291 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.075272    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"
Sep 18 18:06:09.195621 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.195589    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/947b2e8f-8ba2-4912-b0f2-f59b76e3f62e-plugins-config-volume\") pod \"947b2e8f-8ba2-4912-b0f2-f59b76e3f62e\" (UID: \"947b2e8f-8ba2-4912-b0f2-f59b76e3f62e\") "
Sep 18 18:06:09.195811 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.195653    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be0f070a-70fd-4268-a019-a45f147315e8-plugins-config-volume\") pod \"be0f070a-70fd-4268-a019-a45f147315e8\" (UID: \"be0f070a-70fd-4268-a019-a45f147315e8\") "
Sep 18 18:06:09.195811 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.195718    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgp8q\" (UniqueName: \"kubernetes.io/projected/be0f070a-70fd-4268-a019-a45f147315e8-kube-api-access-cgp8q\") pod \"be0f070a-70fd-4268-a019-a45f147315e8\" (UID: \"be0f070a-70fd-4268-a019-a45f147315e8\") "
Sep 18 18:06:09.195811 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.195769    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9c9cz\" (UniqueName: \"kubernetes.io/projected/947b2e8f-8ba2-4912-b0f2-f59b76e3f62e-kube-api-access-9c9cz\") pod \"947b2e8f-8ba2-4912-b0f2-f59b76e3f62e\" (UID: \"947b2e8f-8ba2-4912-b0f2-f59b76e3f62e\") "
Sep 18 18:06:09.196071 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.196043    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/947b2e8f-8ba2-4912-b0f2-f59b76e3f62e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "947b2e8f-8ba2-4912-b0f2-f59b76e3f62e" (UID: "947b2e8f-8ba2-4912-b0f2-f59b76e3f62e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:06:09.196132 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.196071    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be0f070a-70fd-4268-a019-a45f147315e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "be0f070a-70fd-4268-a019-a45f147315e8" (UID: "be0f070a-70fd-4268-a019-a45f147315e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:06:09.197840 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.197819    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/947b2e8f-8ba2-4912-b0f2-f59b76e3f62e-kube-api-access-9c9cz" (OuterVolumeSpecName: "kube-api-access-9c9cz") pod "947b2e8f-8ba2-4912-b0f2-f59b76e3f62e" (UID: "947b2e8f-8ba2-4912-b0f2-f59b76e3f62e"). InnerVolumeSpecName "kube-api-access-9c9cz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:06:09.197904 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.197872    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be0f070a-70fd-4268-a019-a45f147315e8-kube-api-access-cgp8q" (OuterVolumeSpecName: "kube-api-access-cgp8q") pod "be0f070a-70fd-4268-a019-a45f147315e8" (UID: "be0f070a-70fd-4268-a019-a45f147315e8"). InnerVolumeSpecName "kube-api-access-cgp8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:06:09.297368 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.297288    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgp8q\" (UniqueName: \"kubernetes.io/projected/be0f070a-70fd-4268-a019-a45f147315e8-kube-api-access-cgp8q\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:09.297368 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.297319    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c9cz\" (UniqueName: \"kubernetes.io/projected/947b2e8f-8ba2-4912-b0f2-f59b76e3f62e-kube-api-access-9c9cz\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:09.297368 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.297331    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/947b2e8f-8ba2-4912-b0f2-f59b76e3f62e-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:09.297368 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:09.297340    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be0f070a-70fd-4268-a019-a45f147315e8-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:10.041988 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:10.041956    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"
Sep 18 18:06:10.044330 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:10.044305    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"
Sep 18 18:06:10.047003 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:10.046971    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr" event={"ID":"be0f070a-70fd-4268-a019-a45f147315e8","Type":"ContainerDied","Data":"71eb1899ca9cb9b726bc1f9b9bcc3ee03674585dc1f03662371076c42109afbe"}
Sep 18 18:06:10.047126 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:10.047027    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8" event={"ID":"947b2e8f-8ba2-4912-b0f2-f59b76e3f62e","Type":"ContainerDied","Data":"8f45acabebe24591b3f5499e873f5bff3247671a7702ea207ae6b8dd14648773"}
Sep 18 18:06:10.047126 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:10.047061    2568 scope.go:117] "RemoveContainer" containerID="afa1b06b50987927f3dda4be8f56702ab79478416f6eca43dce8082448252470"
Sep 18 18:06:10.059303 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:10.059233    2568 scope.go:117] "RemoveContainer" containerID="bcc1dc0fc21b0b229c096f0702c90ccd5886101d40f9224c8e144a07e3e8b102"
Sep 18 18:06:10.110001 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:10.109966    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"]
Sep 18 18:06:10.151703 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:10.151671    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-a39784-767b6c4678-n7qz8"]
Sep 18 18:06:10.193825 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:10.193788    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"]
Sep 18 18:06:10.204483 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:10.204450    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-a39784-7f855f565c-l9hsr"]
Sep 18 18:06:12.045412 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:12.045337    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="947b2e8f-8ba2-4912-b0f2-f59b76e3f62e" path="/var/lib/kubelet/pods/947b2e8f-8ba2-4912-b0f2-f59b76e3f62e/volumes"
Sep 18 18:06:12.045842 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:12.045691    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be0f070a-70fd-4268-a019-a45f147315e8" path="/var/lib/kubelet/pods/be0f070a-70fd-4268-a019-a45f147315e8/volumes"
Sep 18 18:06:22.663273 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:22.663239    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:06:25.787753 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:25.787720    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"]
Sep 18 18:06:25.788238 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:25.787904    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5" podUID="3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6" containerName="maas-api" containerID="cri-o://9217056368f88a1eca6464fd44f81eb1f1b7e9989c62738162386122fa8287bf" gracePeriod=30
Sep 18 18:06:26.044673 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.044604    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:06:26.133468 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.133431    2568 generic.go:356] "Generic (PLEG): container finished" podID="3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6" containerID="9217056368f88a1eca6464fd44f81eb1f1b7e9989c62738162386122fa8287bf" exitCode=0
Sep 18 18:06:26.133633 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.133492    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"
Sep 18 18:06:26.133633 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.133509    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5" event={"ID":"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6","Type":"ContainerDied","Data":"9217056368f88a1eca6464fd44f81eb1f1b7e9989c62738162386122fa8287bf"}
Sep 18 18:06:26.133633 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.133587    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5" event={"ID":"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6","Type":"ContainerDied","Data":"cb2406158e9938d595f97f75fb44bb62813039b0ebb99beb17d689e860cb1b2d"}
Sep 18 18:06:26.133633 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.133610    2568 scope.go:117] "RemoveContainer" containerID="9217056368f88a1eca6464fd44f81eb1f1b7e9989c62738162386122fa8287bf"
Sep 18 18:06:26.142472 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.142454    2568 scope.go:117] "RemoveContainer" containerID="9217056368f88a1eca6464fd44f81eb1f1b7e9989c62738162386122fa8287bf"
Sep 18 18:06:26.142746 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:06:26.142723    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9217056368f88a1eca6464fd44f81eb1f1b7e9989c62738162386122fa8287bf\": container with ID starting with 9217056368f88a1eca6464fd44f81eb1f1b7e9989c62738162386122fa8287bf not found: ID does not exist" containerID="9217056368f88a1eca6464fd44f81eb1f1b7e9989c62738162386122fa8287bf"
Sep 18 18:06:26.142854 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.142753    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9217056368f88a1eca6464fd44f81eb1f1b7e9989c62738162386122fa8287bf"} err="failed to get container status \"9217056368f88a1eca6464fd44f81eb1f1b7e9989c62738162386122fa8287bf\": rpc error: code = NotFound desc = could not find container \"9217056368f88a1eca6464fd44f81eb1f1b7e9989c62738162386122fa8287bf\": container with ID starting with 9217056368f88a1eca6464fd44f81eb1f1b7e9989c62738162386122fa8287bf not found: ID does not exist"
Sep 18 18:06:26.156704 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.156676    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-maas-api-tls\") pod \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") "
Sep 18 18:06:26.156820 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.156726    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2j68f\" (UniqueName: \"kubernetes.io/projected/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-kube-api-access-2j68f\") pod \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") "
Sep 18 18:06:26.156820 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.156781    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-metrics-certs\") pod \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") "
Sep 18 18:06:26.156820 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.156809    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-openshift-service-ca\") pod \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\" (UID: \"3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6\") "
Sep 18 18:06:26.157478 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.157449    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6" (UID: "3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:06:26.159053 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.159026    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-kube-api-access-2j68f" (OuterVolumeSpecName: "kube-api-access-2j68f") pod "3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6" (UID: "3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6"). InnerVolumeSpecName "kube-api-access-2j68f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:06:26.159350 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.159322    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6" (UID: "3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:06:26.159443 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.159345    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6" (UID: "3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:06:26.258782 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.258734    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:26.258782 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.258775    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2j68f\" (UniqueName: \"kubernetes.io/projected/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-kube-api-access-2j68f\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:26.258782 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.258787    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-metrics-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:26.258782 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.258796    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:26.466657 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.466611    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"]
Sep 18 18:06:26.492013 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:26.491978    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-31e0da22-5c754b668b-hk5b5"]
Sep 18 18:06:28.047298 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:28.047266    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6" path="/var/lib/kubelet/pods/3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6/volumes"
Sep 18 18:06:42.466635 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:42.466603    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:06:52.863973 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:52.863913    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"]
Sep 18 18:06:52.864518 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:52.864239    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh" podUID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" containerName="istio-proxy" containerID="cri-o://00fcfc841b20b440f6eec3bf393b57bbe1a14f650a1a7803cc615c134edb35b1" gracePeriod=30
Sep 18 18:06:55.195532 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:55.195493    2568 patch_prober.go:28] interesting pod/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 18:06:55.195963 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:55.195569    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh" podUID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:06:58.128695 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.128640    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:06:58.171489 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.171460    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istiod-ca-cert\") pod \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") "
Sep 18 18:06:58.171636 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.171507    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-data\") pod \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") "
Sep 18 18:06:58.171636 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.171540    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-credential-socket\") pod \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") "
Sep 18 18:06:58.171636 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.171570    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-workload-certs\") pod \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") "
Sep 18 18:06:58.171636 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.171607    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-workload-socket\") pod \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") "
Sep 18 18:06:58.171636 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.171627    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-envoy\") pod \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") "
Sep 18 18:06:58.171965 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.171676    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-token\") pod \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") "
Sep 18 18:06:58.171965 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.171719    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-podinfo\") pod \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") "
Sep 18 18:06:58.171965 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.171876    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-data" (OuterVolumeSpecName: "istio-data") pod "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" (UID: "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:06:58.171965 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.171905    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l25wr\" (UniqueName: \"kubernetes.io/projected/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-kube-api-access-l25wr\") pod \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\" (UID: \"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209\") "
Sep 18 18:06:58.171965 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.171902    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" (UID: "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:06:58.171965 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.171915    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" (UID: "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:06:58.171965 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.171927    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" (UID: "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:06:58.172359 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.172279    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:58.172359 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.172305    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:58.172359 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.172318    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:58.172359 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.172332    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:58.172359 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.172270    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" (UID: "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:06:58.174072 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.174047    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" (UID: "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:06:58.174256 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.174094    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-kube-api-access-l25wr" (OuterVolumeSpecName: "kube-api-access-l25wr") pod "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" (UID: "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209"). InnerVolumeSpecName "kube-api-access-l25wr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:06:58.174363 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.174343    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-token" (OuterVolumeSpecName: "istio-token") pod "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" (UID: "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:06:58.174422 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.174403    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" (UID: "5a88d247-cfeb-40b7-8b1c-8bfd06fd7209"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:06:58.272991 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.272962    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:58.272991 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.272985    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:58.272991 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.272997    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:58.273271 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.273011    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:58.273271 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.273022    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l25wr\" (UniqueName: \"kubernetes.io/projected/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209-kube-api-access-l25wr\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:06:58.317814 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.317773    2568 generic.go:356] "Generic (PLEG): container finished" podID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" containerID="00fcfc841b20b440f6eec3bf393b57bbe1a14f650a1a7803cc615c134edb35b1" exitCode=0
Sep 18 18:06:58.318013 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.317830    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"
Sep 18 18:06:58.318013 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.317863    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh" event={"ID":"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209","Type":"ContainerDied","Data":"00fcfc841b20b440f6eec3bf393b57bbe1a14f650a1a7803cc615c134edb35b1"}
Sep 18 18:06:58.318013 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.317917    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh" event={"ID":"5a88d247-cfeb-40b7-8b1c-8bfd06fd7209","Type":"ContainerDied","Data":"69a1493cfdf34eeba26af88d604fbca7f562ce0a25227394cf6d21158b3d32b6"}
Sep 18 18:06:58.318013 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.317942    2568 scope.go:117] "RemoveContainer" containerID="00fcfc841b20b440f6eec3bf393b57bbe1a14f650a1a7803cc615c134edb35b1"
Sep 18 18:06:58.328475 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.328458    2568 scope.go:117] "RemoveContainer" containerID="5b5a0e68f662772861cbdb229b9a653c5b37a424a5fd860f441280a0487187c1"
Sep 18 18:06:58.337212 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.337170    2568 scope.go:117] "RemoveContainer" containerID="00fcfc841b20b440f6eec3bf393b57bbe1a14f650a1a7803cc615c134edb35b1"
Sep 18 18:06:58.337465 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:06:58.337448    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00fcfc841b20b440f6eec3bf393b57bbe1a14f650a1a7803cc615c134edb35b1\": container with ID starting with 00fcfc841b20b440f6eec3bf393b57bbe1a14f650a1a7803cc615c134edb35b1 not found: ID does not exist" containerID="00fcfc841b20b440f6eec3bf393b57bbe1a14f650a1a7803cc615c134edb35b1"
Sep 18 18:06:58.337530 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.337472    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00fcfc841b20b440f6eec3bf393b57bbe1a14f650a1a7803cc615c134edb35b1"} err="failed to get container status \"00fcfc841b20b440f6eec3bf393b57bbe1a14f650a1a7803cc615c134edb35b1\": rpc error: code = NotFound desc = could not find container \"00fcfc841b20b440f6eec3bf393b57bbe1a14f650a1a7803cc615c134edb35b1\": container with ID starting with 00fcfc841b20b440f6eec3bf393b57bbe1a14f650a1a7803cc615c134edb35b1 not found: ID does not exist"
Sep 18 18:06:58.337530 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.337488    2568 scope.go:117] "RemoveContainer" containerID="5b5a0e68f662772861cbdb229b9a653c5b37a424a5fd860f441280a0487187c1"
Sep 18 18:06:58.337709 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:06:58.337692    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b5a0e68f662772861cbdb229b9a653c5b37a424a5fd860f441280a0487187c1\": container with ID starting with 5b5a0e68f662772861cbdb229b9a653c5b37a424a5fd860f441280a0487187c1 not found: ID does not exist" containerID="5b5a0e68f662772861cbdb229b9a653c5b37a424a5fd860f441280a0487187c1"
Sep 18 18:06:58.337753 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.337715    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b5a0e68f662772861cbdb229b9a653c5b37a424a5fd860f441280a0487187c1"} err="failed to get container status \"5b5a0e68f662772861cbdb229b9a653c5b37a424a5fd860f441280a0487187c1\": rpc error: code = NotFound desc = could not find container \"5b5a0e68f662772861cbdb229b9a653c5b37a424a5fd860f441280a0487187c1\": container with ID starting with 5b5a0e68f662772861cbdb229b9a653c5b37a424a5fd860f441280a0487187c1 not found: ID does not exist"
Sep 18 18:06:58.346516 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.346494    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"]
Sep 18 18:06:58.353575 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:06:58.353554    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-a39784-openshift-default-6bb5d46758-f9wjh"]
Sep 18 18:07:00.044931 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:00.044901    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" path="/var/lib/kubelet/pods/5a88d247-cfeb-40b7-8b1c-8bfd06fd7209/volumes"
Sep 18 18:07:01.730720 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.730684    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"]
Sep 18 18:07:01.731136 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.731125    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="947b2e8f-8ba2-4912-b0f2-f59b76e3f62e" containerName="payload-pre-processing"
Sep 18 18:07:01.731197 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.731138    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="947b2e8f-8ba2-4912-b0f2-f59b76e3f62e" containerName="payload-pre-processing"
Sep 18 18:07:01.731197 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.731145    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be0f070a-70fd-4268-a019-a45f147315e8" containerName="payload-processing"
Sep 18 18:07:01.731197 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.731152    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="be0f070a-70fd-4268-a019-a45f147315e8" containerName="payload-processing"
Sep 18 18:07:01.731197 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.731166    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" containerName="istio-proxy"
Sep 18 18:07:01.731197 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.731172    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" containerName="istio-proxy"
Sep 18 18:07:01.731388 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.731201    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" containerName="istio-proxy"
Sep 18 18:07:01.731388 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.731206    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" containerName="istio-proxy"
Sep 18 18:07:01.731388 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.731214    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6" containerName="maas-api"
Sep 18 18:07:01.731388 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.731219    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6" containerName="maas-api"
Sep 18 18:07:01.731388 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.731286    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b27d75b-fd05-44f7-87c5-a5c42aa8fbc6" containerName="maas-api"
Sep 18 18:07:01.731388 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.731294    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="947b2e8f-8ba2-4912-b0f2-f59b76e3f62e" containerName="payload-pre-processing"
Sep 18 18:07:01.731388 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.731301    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" containerName="istio-proxy"
Sep 18 18:07:01.731388 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.731309    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="be0f070a-70fd-4268-a019-a45f147315e8" containerName="payload-processing"
Sep 18 18:07:01.735976 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.735955    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:01.744013 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.743992    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-ef5655-serving-cert\""
Sep 18 18:07:01.755925 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.755894    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"]
Sep 18 18:07:01.810836 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.810798    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4f46\" (UniqueName: \"kubernetes.io/projected/d009df22-fe7f-444f-bcee-4eba3eea7d8f-kube-api-access-f4f46\") pod \"maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:01.810836 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.810843    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d009df22-fe7f-444f-bcee-4eba3eea7d8f-openshift-service-ca\") pod \"maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:01.811107 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.810954    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d009df22-fe7f-444f-bcee-4eba3eea7d8f-maas-api-tls\") pod \"maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:01.811107 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.811015    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d009df22-fe7f-444f-bcee-4eba3eea7d8f-metrics-certs\") pod \"maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:01.912438 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.912396    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4f46\" (UniqueName: \"kubernetes.io/projected/d009df22-fe7f-444f-bcee-4eba3eea7d8f-kube-api-access-f4f46\") pod \"maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:01.912654 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.912455    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d009df22-fe7f-444f-bcee-4eba3eea7d8f-openshift-service-ca\") pod \"maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:01.912654 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.912543    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d009df22-fe7f-444f-bcee-4eba3eea7d8f-maas-api-tls\") pod \"maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:01.912654 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.912600    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d009df22-fe7f-444f-bcee-4eba3eea7d8f-metrics-certs\") pod \"maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:01.912819 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:07:01.912690    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-serving-cert: secret "maas-api-e2e-explicit-ef5655-serving-cert" not found
Sep 18 18:07:01.912819 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:07:01.912771    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d009df22-fe7f-444f-bcee-4eba3eea7d8f-maas-api-tls podName:d009df22-fe7f-444f-bcee-4eba3eea7d8f nodeName:}" failed. No retries permitted until 2026-09-18 18:07:02.412746814 +0000 UTC m=+1026.490550714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d009df22-fe7f-444f-bcee-4eba3eea7d8f-maas-api-tls") pod "maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r" (UID: "d009df22-fe7f-444f-bcee-4eba3eea7d8f") : secret "maas-api-e2e-explicit-ef5655-serving-cert" not found
Sep 18 18:07:01.913276 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.913257    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d009df22-fe7f-444f-bcee-4eba3eea7d8f-openshift-service-ca\") pod \"maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:01.915149 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.915131    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d009df22-fe7f-444f-bcee-4eba3eea7d8f-metrics-certs\") pod \"maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:01.929124 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:01.929095    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4f46\" (UniqueName: \"kubernetes.io/projected/d009df22-fe7f-444f-bcee-4eba3eea7d8f-kube-api-access-f4f46\") pod \"maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:02.419101 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:02.419062    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d009df22-fe7f-444f-bcee-4eba3eea7d8f-maas-api-tls\") pod \"maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:02.421710 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:02.421679    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d009df22-fe7f-444f-bcee-4eba3eea7d8f-maas-api-tls\") pod \"maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:02.647395 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:02.647351    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:02.946256 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:02.946224    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"]
Sep 18 18:07:02.948231 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:07:02.948204    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd009df22_fe7f_444f_bcee_4eba3eea7d8f.slice/crio-042b0c63283ce76b8d6f3bf7dc7c219f241cddebb04cc2a1efb0fe2f3bf724a1 WatchSource:0}: Error finding container 042b0c63283ce76b8d6f3bf7dc7c219f241cddebb04cc2a1efb0fe2f3bf724a1: Status 404 returned error can't find the container with id 042b0c63283ce76b8d6f3bf7dc7c219f241cddebb04cc2a1efb0fe2f3bf724a1
Sep 18 18:07:02.949571 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:02.949548    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:07:03.347015 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:03.346922    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r" event={"ID":"d009df22-fe7f-444f-bcee-4eba3eea7d8f","Type":"ContainerStarted","Data":"042b0c63283ce76b8d6f3bf7dc7c219f241cddebb04cc2a1efb0fe2f3bf724a1"}
Sep 18 18:07:04.353835 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:04.353793    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r" event={"ID":"d009df22-fe7f-444f-bcee-4eba3eea7d8f","Type":"ContainerStarted","Data":"aed83ce8a5d1f5fe72f5ca4a1f68883b5170f515d0946f168e971907e533cc26"}
Sep 18 18:07:04.354351 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:04.353869    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:04.381591 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:04.381540    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r" podStartSLOduration=2.665737639 podStartE2EDuration="3.381522652s" totalImagesPullingTime="715.785013ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:07:04.381320688 +0000 UTC m=+1028.459124587" watchObservedRunningTime="2026-09-18 18:07:04.381522652 +0000 UTC m=+1028.459326550"
Sep 18 18:07:09.188624 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:09.188578    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:07:10.365688 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:10.365659    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:07:22.764576 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.764543    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"]
Sep 18 18:07:22.765075 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.765059    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a88d247-cfeb-40b7-8b1c-8bfd06fd7209" containerName="istio-proxy"
Sep 18 18:07:22.768120 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.768102    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.774620 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.774601    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-03e512\"/\"premium-w4-03e512-kserve-self-signed-certs\""
Sep 18 18:07:22.774620 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.774615    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-03e512\"/\"default-dockercfg-wf7xj\""
Sep 18 18:07:22.774783 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.774619    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-03e512\"/\"kube-root-ca.crt\""
Sep 18 18:07:22.774783 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.774653    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-03e512\"/\"openshift-service-ca.crt\""
Sep 18 18:07:22.797506 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.797479    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"]
Sep 18 18:07:22.822058 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.822031    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-home\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.822202 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.822062    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-dshm\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.822202 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.822130    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2djx\" (UniqueName: \"kubernetes.io/projected/efa321cb-a95c-4b6b-9602-06c67d6aaa72-kube-api-access-w2djx\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.822202 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.822155    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-tmp-dir\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.822202 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.822200    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-model-cache\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.822351 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.822249    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa321cb-a95c-4b6b-9602-06c67d6aaa72-tls-certs\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.923271 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.923242    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-home\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.923271 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.923271    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-dshm\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.923449 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.923401    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2djx\" (UniqueName: \"kubernetes.io/projected/efa321cb-a95c-4b6b-9602-06c67d6aaa72-kube-api-access-w2djx\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.923449 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.923441    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-tmp-dir\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.923549 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.923474    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-model-cache\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.923549 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.923503    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa321cb-a95c-4b6b-9602-06c67d6aaa72-tls-certs\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.923691 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.923666    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-home\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.923793 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.923776    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-tmp-dir\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.923875 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.923852    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-model-cache\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.925597 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.925576    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-dshm\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.925947 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.925932    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa321cb-a95c-4b6b-9602-06c67d6aaa72-tls-certs\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:22.932695 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:22.932672    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2djx\" (UniqueName: \"kubernetes.io/projected/efa321cb-a95c-4b6b-9602-06c67d6aaa72-kube-api-access-w2djx\") pod \"premium-w4-03e512-kserve-ccdf875b-6f8jl\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") " pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:23.078224 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:23.078134    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:23.364737 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:23.364710    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"]
Sep 18 18:07:23.367406 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:07:23.367379    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podefa321cb_a95c_4b6b_9602_06c67d6aaa72.slice/crio-5fdee25ed2dcaf2e3b72719ddf384a37800042d61450aa2a6c985cabeb06b63b WatchSource:0}: Error finding container 5fdee25ed2dcaf2e3b72719ddf384a37800042d61450aa2a6c985cabeb06b63b: Status 404 returned error can't find the container with id 5fdee25ed2dcaf2e3b72719ddf384a37800042d61450aa2a6c985cabeb06b63b
Sep 18 18:07:23.462115 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:23.462080    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl" event={"ID":"efa321cb-a95c-4b6b-9602-06c67d6aaa72","Type":"ContainerStarted","Data":"1ad2731fdefb487fee875df132e9af7c8898ceb937f324f2274f079640167883"}
Sep 18 18:07:23.462287 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:23.462132    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl" event={"ID":"efa321cb-a95c-4b6b-9602-06c67d6aaa72","Type":"ContainerStarted","Data":"5fdee25ed2dcaf2e3b72719ddf384a37800042d61450aa2a6c985cabeb06b63b"}
Sep 18 18:07:23.496303 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:23.496258    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl" podStartSLOduration=1.496242797 podStartE2EDuration="1.496242797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:07:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:07:23.493235353 +0000 UTC m=+1047.571039249" watchObservedRunningTime="2026-09-18 18:07:23.496242797 +0000 UTC m=+1047.574046695"
Sep 18 18:07:29.461272 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:29.461235    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:07:33.079092 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:33.079057    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:33.079092 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:33.079102    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:33.091590 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:33.091560    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:33.529090 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:33.529062    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:07:51.251228 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:07:51.251124    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:08:07.716103 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.716068    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"]
Sep 18 18:08:07.721021 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.721004    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.725069 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.725048    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ee3049\"/\"openshift-service-ca.crt\""
Sep 18 18:08:07.725069 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.725063    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ee3049\"/\"kube-root-ca.crt\""
Sep 18 18:08:07.725885 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.725865    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ee3049\"/\"test-model-ee3049-kserve-self-signed-certs\""
Sep 18 18:08:07.726136 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.726116    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ee3049\"/\"default-dockercfg-gs8zq\""
Sep 18 18:08:07.748442 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.748419    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"]
Sep 18 18:08:07.844606 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.844576    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-dshm\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.844759 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.844630    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-tmp-dir\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.844759 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.844714    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctkr6\" (UniqueName: \"kubernetes.io/projected/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-kube-api-access-ctkr6\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.844759 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.844753    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-model-cache\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.844923 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.844784    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-tls-certs\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.844923 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.844826    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-home\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.945482 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.945448    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-tmp-dir\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.945656 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.945497    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctkr6\" (UniqueName: \"kubernetes.io/projected/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-kube-api-access-ctkr6\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.945656 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.945523    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-model-cache\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.945780 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.945688    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-tls-certs\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.945780 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.945739    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-home\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.945873 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.945783    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-dshm\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.945873 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.945847    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-tmp-dir\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.945955 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.945855    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-model-cache\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.946122 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.946102    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-home\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.948001 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.947979    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-dshm\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.948224 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.948200    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-tls-certs\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:07.960139 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:07.960114    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctkr6\" (UniqueName: \"kubernetes.io/projected/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-kube-api-access-ctkr6\") pod \"test-model-ee3049-kserve-ccff6c98f-rhjgl\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") " pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:08.030826 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:08.030742    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:08.342522 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:08.342499    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"]
Sep 18 18:08:08.343353 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:08:08.343331    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f0fcaaf_adff_4b5a_a0ad_154ebc8ffee1.slice/crio-6c6273c6638b76558f8500ff80e83523dd2a3537bc5b87cd6204f824790745bd WatchSource:0}: Error finding container 6c6273c6638b76558f8500ff80e83523dd2a3537bc5b87cd6204f824790745bd: Status 404 returned error can't find the container with id 6c6273c6638b76558f8500ff80e83523dd2a3537bc5b87cd6204f824790745bd
Sep 18 18:08:08.718159 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:08.718123    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl" event={"ID":"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1","Type":"ContainerStarted","Data":"722306262f91cac9024a31bff8fdb7abc7de623434e4c3b600256ef2fa419936"}
Sep 18 18:08:08.718563 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:08.718169    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl" event={"ID":"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1","Type":"ContainerStarted","Data":"6c6273c6638b76558f8500ff80e83523dd2a3537bc5b87cd6204f824790745bd"}
Sep 18 18:08:08.762944 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:08.762880    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl" podStartSLOduration=1.762859548 podStartE2EDuration="1.762859548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:08:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:08:08.759720675 +0000 UTC m=+1092.837524579" watchObservedRunningTime="2026-09-18 18:08:08.762859548 +0000 UTC m=+1092.840663451"
Sep 18 18:08:14.941570 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:14.941477    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:08:14.973533 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:14.973501    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"]
Sep 18 18:08:14.977349 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:14.977331    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:14.992097 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:14.992045    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3ffa634f-openshift-default-dockercfg-2vw47\""
Sep 18 18:08:14.992334 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:14.992310    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 18:08:15.012771 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.012725    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-data\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.012771 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.012768    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-token\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.013051 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.012815    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-podinfo\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.013051 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.012846    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69f5d2a2-f501-4adc-9a78-88ab2449895b-istiod-ca-cert\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.013051 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.012906    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-workload-socket\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.013051 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.012934    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-credential-socket\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.013051 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.013042    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-envoy\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.013375 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.013077    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6cmb\" (UniqueName: \"kubernetes.io/projected/69f5d2a2-f501-4adc-9a78-88ab2449895b-kube-api-access-m6cmb\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.013375 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.013135    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-workload-certs\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.016397 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.016370    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"]
Sep 18 18:08:15.113902 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.113869    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-token\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.113902 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.113909    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-podinfo\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.114196 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.113928    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69f5d2a2-f501-4adc-9a78-88ab2449895b-istiod-ca-cert\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.114196 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.113979    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-workload-socket\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.114196 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.114003    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-credential-socket\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.114196 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.114098    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-envoy\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.114196 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.114142    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6cmb\" (UniqueName: \"kubernetes.io/projected/69f5d2a2-f501-4adc-9a78-88ab2449895b-kube-api-access-m6cmb\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.114485 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.114252    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-workload-certs\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.114485 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.114350    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-data\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.114593 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.114553    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-workload-socket\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.114645 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.114586    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-credential-socket\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.114699 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.114681    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-data\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.114799 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.114778    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-workload-certs\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.114979 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.114959    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69f5d2a2-f501-4adc-9a78-88ab2449895b-istiod-ca-cert\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.116583 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.116560    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-envoy\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.116758 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.116737    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-podinfo\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.154953 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.154922    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-token\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.155231 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.155209    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6cmb\" (UniqueName: \"kubernetes.io/projected/69f5d2a2-f501-4adc-9a78-88ab2449895b-kube-api-access-m6cmb\") pod \"e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") " pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.290015 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.289920    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:15.872834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.872807    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"]
Sep 18 18:08:15.875450 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:08:15.875418    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69f5d2a2_f501_4adc_9a78_88ab2449895b.slice/crio-bdd2e70f97199081813625ed9e6b93ef29a8887c2056d1bb15b6ca26b69c0652 WatchSource:0}: Error finding container bdd2e70f97199081813625ed9e6b93ef29a8887c2056d1bb15b6ca26b69c0652: Status 404 returned error can't find the container with id bdd2e70f97199081813625ed9e6b93ef29a8887c2056d1bb15b6ca26b69c0652
Sep 18 18:08:15.877649 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.877611    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:08:15.877738 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.877680    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:08:15.877738 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:15.877708    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:08:16.769342 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:16.769304    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr" event={"ID":"69f5d2a2-f501-4adc-9a78-88ab2449895b","Type":"ContainerStarted","Data":"0f58f57168185f5edde68f15d58e940fe3e847578bcb363ed946543a9c2e20a9"}
Sep 18 18:08:16.769745 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:16.769360    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr" event={"ID":"69f5d2a2-f501-4adc-9a78-88ab2449895b","Type":"ContainerStarted","Data":"bdd2e70f97199081813625ed9e6b93ef29a8887c2056d1bb15b6ca26b69c0652"}
Sep 18 18:08:17.290311 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:17.290269    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:17.296499 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:17.296467    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:17.616272 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:17.616143    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr" podStartSLOduration=3.6161244200000002 podStartE2EDuration="3.61612442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:08:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:08:17.088906472 +0000 UTC m=+1101.166710372" watchObservedRunningTime="2026-09-18 18:08:17.61612442 +0000 UTC m=+1101.693928320"
Sep 18 18:08:17.775542 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:17.775512    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:17.776493 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:17.776472    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:18.030873 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:18.030837    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:18.031089 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:18.030898    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:18.046538 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:18.046514    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:18.792072 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:18.792047    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:08:36.747807 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.747773    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"]
Sep 18 18:08:36.751736 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.751720    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:36.757882 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.757858    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-3ffa634f-serving-cert\""
Sep 18 18:08:36.783654 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.783623    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"]
Sep 18 18:08:36.834524 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.834493    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a53b6b43-9321-47f6-a7a3-a54735d7d46d-openshift-service-ca\") pod \"maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:36.834710 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.834539    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a53b6b43-9321-47f6-a7a3-a54735d7d46d-metrics-certs\") pod \"maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:36.834710 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.834593    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a53b6b43-9321-47f6-a7a3-a54735d7d46d-maas-api-tls\") pod \"maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:36.834710 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.834688    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mvml\" (UniqueName: \"kubernetes.io/projected/a53b6b43-9321-47f6-a7a3-a54735d7d46d-kube-api-access-9mvml\") pod \"maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:36.844006 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.843979    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:08:36.936142 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.936108    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a53b6b43-9321-47f6-a7a3-a54735d7d46d-metrics-certs\") pod \"maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:36.936325 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.936158    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a53b6b43-9321-47f6-a7a3-a54735d7d46d-maas-api-tls\") pod \"maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:36.936325 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.936255    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mvml\" (UniqueName: \"kubernetes.io/projected/a53b6b43-9321-47f6-a7a3-a54735d7d46d-kube-api-access-9mvml\") pod \"maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:36.936325 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.936302    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a53b6b43-9321-47f6-a7a3-a54735d7d46d-openshift-service-ca\") pod \"maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:36.936492 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:08:36.936380    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-serving-cert: secret "maas-api-e2e-mt-3ffa634f-serving-cert" not found
Sep 18 18:08:36.936546 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:08:36.936506    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a53b6b43-9321-47f6-a7a3-a54735d7d46d-maas-api-tls podName:a53b6b43-9321-47f6-a7a3-a54735d7d46d nodeName:}" failed. No retries permitted until 2026-09-18 18:08:37.436484457 +0000 UTC m=+1121.514288348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a53b6b43-9321-47f6-a7a3-a54735d7d46d-maas-api-tls") pod "maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf" (UID: "a53b6b43-9321-47f6-a7a3-a54735d7d46d") : secret "maas-api-e2e-mt-3ffa634f-serving-cert" not found
Sep 18 18:08:36.937086 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.937060    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a53b6b43-9321-47f6-a7a3-a54735d7d46d-openshift-service-ca\") pod \"maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:36.938700 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.938680    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a53b6b43-9321-47f6-a7a3-a54735d7d46d-metrics-certs\") pod \"maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:36.958711 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:36.958687    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mvml\" (UniqueName: \"kubernetes.io/projected/a53b6b43-9321-47f6-a7a3-a54735d7d46d-kube-api-access-9mvml\") pod \"maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:37.442290 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:37.442244    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a53b6b43-9321-47f6-a7a3-a54735d7d46d-maas-api-tls\") pod \"maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:37.444664 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:37.444644    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a53b6b43-9321-47f6-a7a3-a54735d7d46d-maas-api-tls\") pod \"maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:37.664110 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:37.664073    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:37.998427 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:37.998351    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"]
Sep 18 18:08:38.005282 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:08:38.005257    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda53b6b43_9321_47f6_a7a3_a54735d7d46d.slice/crio-6ee6e95af21f2920561cbead292db9d40d04b5d0e027ab38b8529fb4eb880158 WatchSource:0}: Error finding container 6ee6e95af21f2920561cbead292db9d40d04b5d0e027ab38b8529fb4eb880158: Status 404 returned error can't find the container with id 6ee6e95af21f2920561cbead292db9d40d04b5d0e027ab38b8529fb4eb880158
Sep 18 18:08:38.894832 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:38.894799    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf" event={"ID":"a53b6b43-9321-47f6-a7a3-a54735d7d46d","Type":"ContainerStarted","Data":"f1c72b65c8c612dbfda5e97b72b695d7eb9093b32563d27de58fff3955cf3fab"}
Sep 18 18:08:38.895108 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:38.894853    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf" event={"ID":"a53b6b43-9321-47f6-a7a3-a54735d7d46d","Type":"ContainerStarted","Data":"6ee6e95af21f2920561cbead292db9d40d04b5d0e027ab38b8529fb4eb880158"}
Sep 18 18:08:38.895108 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:38.894905    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:44.906228 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:44.906165    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:08:44.939036 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:44.938986    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf" podStartSLOduration=8.546638252 podStartE2EDuration="8.938971801s" totalImagesPullingTime="392.333549ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:08:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:08:38.948677737 +0000 UTC m=+1123.026481636" watchObservedRunningTime="2026-09-18 18:08:44.938971801 +0000 UTC m=+1129.016775738"
Sep 18 18:08:57.006414 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:57.006379    2568 generic.go:356] "Generic (PLEG): container finished" podID="69f5d2a2-f501-4adc-9a78-88ab2449895b" containerID="0f58f57168185f5edde68f15d58e940fe3e847578bcb363ed946543a9c2e20a9" exitCode=0
Sep 18 18:08:57.006835 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:57.006457    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr" event={"ID":"69f5d2a2-f501-4adc-9a78-88ab2449895b","Type":"ContainerDied","Data":"0f58f57168185f5edde68f15d58e940fe3e847578bcb363ed946543a9c2e20a9"}
Sep 18 18:08:57.006909 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:57.006894    2568 scope.go:117] "RemoveContainer" containerID="0f58f57168185f5edde68f15d58e940fe3e847578bcb363ed946543a9c2e20a9"
Sep 18 18:08:57.015114 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:57.008491    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:08:57.015114 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:57.008699    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:08:57.015114 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:57.008799    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:08:58.014166 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:58.014129    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr" event={"ID":"69f5d2a2-f501-4adc-9a78-88ab2449895b","Type":"ContainerStarted","Data":"1cd752af89b2208bb73947ec3e4667745cd2d42b2e418f931fea1380f566e0f5"}
Sep 18 18:08:58.291097 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:58.291005    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:08:58.292451 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:58.292424    2568 patch_prober.go:28] interesting pod/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Sep 18 18:08:58.292575 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:58.292473    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr" podUID="69f5d2a2-f501-4adc-9a78-88ab2449895b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Sep 18 18:08:59.290949 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:59.290911    2568 patch_prober.go:28] interesting pod/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Sep 18 18:08:59.291353 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:08:59.290968    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr" podUID="69f5d2a2-f501-4adc-9a78-88ab2449895b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Sep 18 18:09:00.291381 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:00.291341    2568 patch_prober.go:28] interesting pod/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Sep 18 18:09:00.291740 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:00.291398    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr" podUID="69f5d2a2-f501-4adc-9a78-88ab2449895b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Sep 18 18:09:01.290385 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:01.290341    2568 patch_prober.go:28] interesting pod/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Sep 18 18:09:01.290571 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:01.290391    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr" podUID="69f5d2a2-f501-4adc-9a78-88ab2449895b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Sep 18 18:09:02.291114 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:02.291067    2568 patch_prober.go:28] interesting pod/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Sep 18 18:09:02.291572 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:02.291138    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr" podUID="69f5d2a2-f501-4adc-9a78-88ab2449895b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Sep 18 18:09:02.776289 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:02.776238    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:09:03.375033 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:03.375001    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:09:03.377393 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:03.377370    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:09:06.225277 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:06.225239    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:09:23.287189 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:23.287089    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"]
Sep 18 18:09:23.287651 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:23.287350    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf" podUID="a53b6b43-9321-47f6-a7a3-a54735d7d46d" containerName="maas-api" containerID="cri-o://f1c72b65c8c612dbfda5e97b72b695d7eb9093b32563d27de58fff3955cf3fab" gracePeriod=30
Sep 18 18:09:23.545340 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:23.545274    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:09:23.636597 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:23.636564    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a53b6b43-9321-47f6-a7a3-a54735d7d46d-openshift-service-ca\") pod \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") "
Sep 18 18:09:23.636769 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:23.636631    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a53b6b43-9321-47f6-a7a3-a54735d7d46d-maas-api-tls\") pod \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") "
Sep 18 18:09:23.636769 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:23.636657    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a53b6b43-9321-47f6-a7a3-a54735d7d46d-metrics-certs\") pod \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") "
Sep 18 18:09:23.636769 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:23.636714    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mvml\" (UniqueName: \"kubernetes.io/projected/a53b6b43-9321-47f6-a7a3-a54735d7d46d-kube-api-access-9mvml\") pod \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\" (UID: \"a53b6b43-9321-47f6-a7a3-a54735d7d46d\") "
Sep 18 18:09:23.637007 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:23.636984    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a53b6b43-9321-47f6-a7a3-a54735d7d46d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a53b6b43-9321-47f6-a7a3-a54735d7d46d" (UID: "a53b6b43-9321-47f6-a7a3-a54735d7d46d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:09:23.637080 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:23.637065    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a53b6b43-9321-47f6-a7a3-a54735d7d46d-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:23.638928 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:23.638899    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a53b6b43-9321-47f6-a7a3-a54735d7d46d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a53b6b43-9321-47f6-a7a3-a54735d7d46d" (UID: "a53b6b43-9321-47f6-a7a3-a54735d7d46d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:09:23.639044 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:23.639017    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a53b6b43-9321-47f6-a7a3-a54735d7d46d-kube-api-access-9mvml" (OuterVolumeSpecName: "kube-api-access-9mvml") pod "a53b6b43-9321-47f6-a7a3-a54735d7d46d" (UID: "a53b6b43-9321-47f6-a7a3-a54735d7d46d"). InnerVolumeSpecName "kube-api-access-9mvml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:09:23.639087 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:23.639019    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a53b6b43-9321-47f6-a7a3-a54735d7d46d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a53b6b43-9321-47f6-a7a3-a54735d7d46d" (UID: "a53b6b43-9321-47f6-a7a3-a54735d7d46d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:09:23.737593 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:23.737553    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mvml\" (UniqueName: \"kubernetes.io/projected/a53b6b43-9321-47f6-a7a3-a54735d7d46d-kube-api-access-9mvml\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:23.737593 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:23.737585    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a53b6b43-9321-47f6-a7a3-a54735d7d46d-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:23.737593 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:23.737594    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a53b6b43-9321-47f6-a7a3-a54735d7d46d-metrics-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:24.158680 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:24.158643    2568 generic.go:356] "Generic (PLEG): container finished" podID="a53b6b43-9321-47f6-a7a3-a54735d7d46d" containerID="f1c72b65c8c612dbfda5e97b72b695d7eb9093b32563d27de58fff3955cf3fab" exitCode=0
Sep 18 18:09:24.158872 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:24.158692    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"
Sep 18 18:09:24.158872 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:24.158730    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf" event={"ID":"a53b6b43-9321-47f6-a7a3-a54735d7d46d","Type":"ContainerDied","Data":"f1c72b65c8c612dbfda5e97b72b695d7eb9093b32563d27de58fff3955cf3fab"}
Sep 18 18:09:24.158872 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:24.158796    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf" event={"ID":"a53b6b43-9321-47f6-a7a3-a54735d7d46d","Type":"ContainerDied","Data":"6ee6e95af21f2920561cbead292db9d40d04b5d0e027ab38b8529fb4eb880158"}
Sep 18 18:09:24.158872 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:24.158823    2568 scope.go:117] "RemoveContainer" containerID="f1c72b65c8c612dbfda5e97b72b695d7eb9093b32563d27de58fff3955cf3fab"
Sep 18 18:09:24.170885 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:24.170841    2568 scope.go:117] "RemoveContainer" containerID="f1c72b65c8c612dbfda5e97b72b695d7eb9093b32563d27de58fff3955cf3fab"
Sep 18 18:09:24.171238 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:09:24.171215    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1c72b65c8c612dbfda5e97b72b695d7eb9093b32563d27de58fff3955cf3fab\": container with ID starting with f1c72b65c8c612dbfda5e97b72b695d7eb9093b32563d27de58fff3955cf3fab not found: ID does not exist" containerID="f1c72b65c8c612dbfda5e97b72b695d7eb9093b32563d27de58fff3955cf3fab"
Sep 18 18:09:24.171328 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:24.171249    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1c72b65c8c612dbfda5e97b72b695d7eb9093b32563d27de58fff3955cf3fab"} err="failed to get container status \"f1c72b65c8c612dbfda5e97b72b695d7eb9093b32563d27de58fff3955cf3fab\": rpc error: code = NotFound desc = could not find container \"f1c72b65c8c612dbfda5e97b72b695d7eb9093b32563d27de58fff3955cf3fab\": container with ID starting with f1c72b65c8c612dbfda5e97b72b695d7eb9093b32563d27de58fff3955cf3fab not found: ID does not exist"
Sep 18 18:09:24.198817 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:24.198786    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"]
Sep 18 18:09:24.200726 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:24.200706    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3ffa634f-78f5cc4bb9-qlwdf"]
Sep 18 18:09:26.045906 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:26.045867    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a53b6b43-9321-47f6-a7a3-a54735d7d46d" path="/var/lib/kubelet/pods/a53b6b43-9321-47f6-a7a3-a54735d7d46d/volumes"
Sep 18 18:09:29.539587 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:29.539552    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:09:30.733851 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:30.733811    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"]
Sep 18 18:09:30.734275 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:30.734080    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr" podUID="69f5d2a2-f501-4adc-9a78-88ab2449895b" containerName="istio-proxy" containerID="cri-o://1cd752af89b2208bb73947ec3e4667745cd2d42b2e418f931fea1380f566e0f5" gracePeriod=30
Sep 18 18:09:33.377712 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:33.377676    2568 patch_prober.go:28] interesting pod/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 18:09:33.378086 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:33.377749    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr" podUID="69f5d2a2-f501-4adc-9a78-88ab2449895b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:09:35.999599 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:35.999541    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:09:36.062208 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.062152    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-podinfo\") pod \"69f5d2a2-f501-4adc-9a78-88ab2449895b\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") "
Sep 18 18:09:36.062381 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.062213    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-data\") pod \"69f5d2a2-f501-4adc-9a78-88ab2449895b\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") "
Sep 18 18:09:36.062381 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.062255    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6cmb\" (UniqueName: \"kubernetes.io/projected/69f5d2a2-f501-4adc-9a78-88ab2449895b-kube-api-access-m6cmb\") pod \"69f5d2a2-f501-4adc-9a78-88ab2449895b\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") "
Sep 18 18:09:36.062381 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.062323    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69f5d2a2-f501-4adc-9a78-88ab2449895b-istiod-ca-cert\") pod \"69f5d2a2-f501-4adc-9a78-88ab2449895b\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") "
Sep 18 18:09:36.062381 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.062369    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-envoy\") pod \"69f5d2a2-f501-4adc-9a78-88ab2449895b\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") "
Sep 18 18:09:36.062606 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.062431    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-credential-socket\") pod \"69f5d2a2-f501-4adc-9a78-88ab2449895b\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") "
Sep 18 18:09:36.062606 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.062462    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-token\") pod \"69f5d2a2-f501-4adc-9a78-88ab2449895b\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") "
Sep 18 18:09:36.062606 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.062507    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-workload-socket\") pod \"69f5d2a2-f501-4adc-9a78-88ab2449895b\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") "
Sep 18 18:09:36.062606 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.062517    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-data" (OuterVolumeSpecName: "istio-data") pod "69f5d2a2-f501-4adc-9a78-88ab2449895b" (UID: "69f5d2a2-f501-4adc-9a78-88ab2449895b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:09:36.062606 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.062544    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-workload-certs\") pod \"69f5d2a2-f501-4adc-9a78-88ab2449895b\" (UID: \"69f5d2a2-f501-4adc-9a78-88ab2449895b\") "
Sep 18 18:09:36.062863 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.062781    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "69f5d2a2-f501-4adc-9a78-88ab2449895b" (UID: "69f5d2a2-f501-4adc-9a78-88ab2449895b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:09:36.062927 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.062891    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:36.062927 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.062911    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:36.063043 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.063009    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "69f5d2a2-f501-4adc-9a78-88ab2449895b" (UID: "69f5d2a2-f501-4adc-9a78-88ab2449895b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:09:36.063131 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.063104    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69f5d2a2-f501-4adc-9a78-88ab2449895b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "69f5d2a2-f501-4adc-9a78-88ab2449895b" (UID: "69f5d2a2-f501-4adc-9a78-88ab2449895b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:09:36.063332 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.063291    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "69f5d2a2-f501-4adc-9a78-88ab2449895b" (UID: "69f5d2a2-f501-4adc-9a78-88ab2449895b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:09:36.064811 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.064783    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "69f5d2a2-f501-4adc-9a78-88ab2449895b" (UID: "69f5d2a2-f501-4adc-9a78-88ab2449895b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:09:36.064924 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.064850    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-token" (OuterVolumeSpecName: "istio-token") pod "69f5d2a2-f501-4adc-9a78-88ab2449895b" (UID: "69f5d2a2-f501-4adc-9a78-88ab2449895b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:09:36.065067 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.065031    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69f5d2a2-f501-4adc-9a78-88ab2449895b-kube-api-access-m6cmb" (OuterVolumeSpecName: "kube-api-access-m6cmb") pod "69f5d2a2-f501-4adc-9a78-88ab2449895b" (UID: "69f5d2a2-f501-4adc-9a78-88ab2449895b"). InnerVolumeSpecName "kube-api-access-m6cmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:09:36.065067 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.065044    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "69f5d2a2-f501-4adc-9a78-88ab2449895b" (UID: "69f5d2a2-f501-4adc-9a78-88ab2449895b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:09:36.164764 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.164732    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:36.164764 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.164763    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:36.164992 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.164777    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:36.164992 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.164790    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69f5d2a2-f501-4adc-9a78-88ab2449895b-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:36.164992 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.164802    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69f5d2a2-f501-4adc-9a78-88ab2449895b-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:36.164992 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.164815    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6cmb\" (UniqueName: \"kubernetes.io/projected/69f5d2a2-f501-4adc-9a78-88ab2449895b-kube-api-access-m6cmb\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:36.164992 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.164829    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69f5d2a2-f501-4adc-9a78-88ab2449895b-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:36.227848 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.227817    2568 generic.go:356] "Generic (PLEG): container finished" podID="69f5d2a2-f501-4adc-9a78-88ab2449895b" containerID="1cd752af89b2208bb73947ec3e4667745cd2d42b2e418f931fea1380f566e0f5" exitCode=0
Sep 18 18:09:36.228032 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.227888    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"
Sep 18 18:09:36.228032 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.227917    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr" event={"ID":"69f5d2a2-f501-4adc-9a78-88ab2449895b","Type":"ContainerDied","Data":"1cd752af89b2208bb73947ec3e4667745cd2d42b2e418f931fea1380f566e0f5"}
Sep 18 18:09:36.228032 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.227977    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr" event={"ID":"69f5d2a2-f501-4adc-9a78-88ab2449895b","Type":"ContainerDied","Data":"bdd2e70f97199081813625ed9e6b93ef29a8887c2056d1bb15b6ca26b69c0652"}
Sep 18 18:09:36.228032 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.228001    2568 scope.go:117] "RemoveContainer" containerID="1cd752af89b2208bb73947ec3e4667745cd2d42b2e418f931fea1380f566e0f5"
Sep 18 18:09:36.238417 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.238401    2568 scope.go:117] "RemoveContainer" containerID="0f58f57168185f5edde68f15d58e940fe3e847578bcb363ed946543a9c2e20a9"
Sep 18 18:09:36.246426 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.246408    2568 scope.go:117] "RemoveContainer" containerID="1cd752af89b2208bb73947ec3e4667745cd2d42b2e418f931fea1380f566e0f5"
Sep 18 18:09:36.246694 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:09:36.246677    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cd752af89b2208bb73947ec3e4667745cd2d42b2e418f931fea1380f566e0f5\": container with ID starting with 1cd752af89b2208bb73947ec3e4667745cd2d42b2e418f931fea1380f566e0f5 not found: ID does not exist" containerID="1cd752af89b2208bb73947ec3e4667745cd2d42b2e418f931fea1380f566e0f5"
Sep 18 18:09:36.246748 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.246702    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cd752af89b2208bb73947ec3e4667745cd2d42b2e418f931fea1380f566e0f5"} err="failed to get container status \"1cd752af89b2208bb73947ec3e4667745cd2d42b2e418f931fea1380f566e0f5\": rpc error: code = NotFound desc = could not find container \"1cd752af89b2208bb73947ec3e4667745cd2d42b2e418f931fea1380f566e0f5\": container with ID starting with 1cd752af89b2208bb73947ec3e4667745cd2d42b2e418f931fea1380f566e0f5 not found: ID does not exist"
Sep 18 18:09:36.246748 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.246719    2568 scope.go:117] "RemoveContainer" containerID="0f58f57168185f5edde68f15d58e940fe3e847578bcb363ed946543a9c2e20a9"
Sep 18 18:09:36.246989 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:09:36.246971    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f58f57168185f5edde68f15d58e940fe3e847578bcb363ed946543a9c2e20a9\": container with ID starting with 0f58f57168185f5edde68f15d58e940fe3e847578bcb363ed946543a9c2e20a9 not found: ID does not exist" containerID="0f58f57168185f5edde68f15d58e940fe3e847578bcb363ed946543a9c2e20a9"
Sep 18 18:09:36.247044 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.246994    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f58f57168185f5edde68f15d58e940fe3e847578bcb363ed946543a9c2e20a9"} err="failed to get container status \"0f58f57168185f5edde68f15d58e940fe3e847578bcb363ed946543a9c2e20a9\": rpc error: code = NotFound desc = could not find container \"0f58f57168185f5edde68f15d58e940fe3e847578bcb363ed946543a9c2e20a9\": container with ID starting with 0f58f57168185f5edde68f15d58e940fe3e847578bcb363ed946543a9c2e20a9 not found: ID does not exist"
Sep 18 18:09:36.271655 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.271589    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"]
Sep 18 18:09:36.273803 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.273784    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3ffa634f-openshift-default-8bfcbb9c6-jlgkr"]
Sep 18 18:09:36.428334 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.428301    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"]
Sep 18 18:09:36.428562 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.428540    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r" podUID="d009df22-fe7f-444f-bcee-4eba3eea7d8f" containerName="maas-api" containerID="cri-o://aed83ce8a5d1f5fe72f5ca4a1f68883b5170f515d0946f168e971907e533cc26" gracePeriod=30
Sep 18 18:09:36.671592 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.671566    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:09:36.771838 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.771805    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4f46\" (UniqueName: \"kubernetes.io/projected/d009df22-fe7f-444f-bcee-4eba3eea7d8f-kube-api-access-f4f46\") pod \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") "
Sep 18 18:09:36.771999 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.771882    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d009df22-fe7f-444f-bcee-4eba3eea7d8f-metrics-certs\") pod \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") "
Sep 18 18:09:36.771999 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.771925    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d009df22-fe7f-444f-bcee-4eba3eea7d8f-openshift-service-ca\") pod \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") "
Sep 18 18:09:36.771999 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.771955    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d009df22-fe7f-444f-bcee-4eba3eea7d8f-maas-api-tls\") pod \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\" (UID: \"d009df22-fe7f-444f-bcee-4eba3eea7d8f\") "
Sep 18 18:09:36.772360 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.772332    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d009df22-fe7f-444f-bcee-4eba3eea7d8f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d009df22-fe7f-444f-bcee-4eba3eea7d8f" (UID: "d009df22-fe7f-444f-bcee-4eba3eea7d8f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:09:36.772480 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.772434    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d009df22-fe7f-444f-bcee-4eba3eea7d8f-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:36.774439 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.774416    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d009df22-fe7f-444f-bcee-4eba3eea7d8f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d009df22-fe7f-444f-bcee-4eba3eea7d8f" (UID: "d009df22-fe7f-444f-bcee-4eba3eea7d8f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:09:36.774590 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.774569    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d009df22-fe7f-444f-bcee-4eba3eea7d8f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d009df22-fe7f-444f-bcee-4eba3eea7d8f" (UID: "d009df22-fe7f-444f-bcee-4eba3eea7d8f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:09:36.774643 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.774597    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d009df22-fe7f-444f-bcee-4eba3eea7d8f-kube-api-access-f4f46" (OuterVolumeSpecName: "kube-api-access-f4f46") pod "d009df22-fe7f-444f-bcee-4eba3eea7d8f" (UID: "d009df22-fe7f-444f-bcee-4eba3eea7d8f"). InnerVolumeSpecName "kube-api-access-f4f46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:09:36.873737 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.873661    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d009df22-fe7f-444f-bcee-4eba3eea7d8f-metrics-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:36.873737 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.873687    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d009df22-fe7f-444f-bcee-4eba3eea7d8f-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:36.873737 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:36.873698    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4f46\" (UniqueName: \"kubernetes.io/projected/d009df22-fe7f-444f-bcee-4eba3eea7d8f-kube-api-access-f4f46\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:09:37.235018 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:37.234984    2568 generic.go:356] "Generic (PLEG): container finished" podID="d009df22-fe7f-444f-bcee-4eba3eea7d8f" containerID="aed83ce8a5d1f5fe72f5ca4a1f68883b5170f515d0946f168e971907e533cc26" exitCode=0
Sep 18 18:09:37.235506 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:37.235034    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"
Sep 18 18:09:37.235506 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:37.235073    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r" event={"ID":"d009df22-fe7f-444f-bcee-4eba3eea7d8f","Type":"ContainerDied","Data":"aed83ce8a5d1f5fe72f5ca4a1f68883b5170f515d0946f168e971907e533cc26"}
Sep 18 18:09:37.235506 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:37.235137    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r" event={"ID":"d009df22-fe7f-444f-bcee-4eba3eea7d8f","Type":"ContainerDied","Data":"042b0c63283ce76b8d6f3bf7dc7c219f241cddebb04cc2a1efb0fe2f3bf724a1"}
Sep 18 18:09:37.235506 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:37.235172    2568 scope.go:117] "RemoveContainer" containerID="aed83ce8a5d1f5fe72f5ca4a1f68883b5170f515d0946f168e971907e533cc26"
Sep 18 18:09:37.245462 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:37.245442    2568 scope.go:117] "RemoveContainer" containerID="aed83ce8a5d1f5fe72f5ca4a1f68883b5170f515d0946f168e971907e533cc26"
Sep 18 18:09:37.245744 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:09:37.245720    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aed83ce8a5d1f5fe72f5ca4a1f68883b5170f515d0946f168e971907e533cc26\": container with ID starting with aed83ce8a5d1f5fe72f5ca4a1f68883b5170f515d0946f168e971907e533cc26 not found: ID does not exist" containerID="aed83ce8a5d1f5fe72f5ca4a1f68883b5170f515d0946f168e971907e533cc26"
Sep 18 18:09:37.245847 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:37.245749    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aed83ce8a5d1f5fe72f5ca4a1f68883b5170f515d0946f168e971907e533cc26"} err="failed to get container status \"aed83ce8a5d1f5fe72f5ca4a1f68883b5170f515d0946f168e971907e533cc26\": rpc error: code = NotFound desc = could not find container \"aed83ce8a5d1f5fe72f5ca4a1f68883b5170f515d0946f168e971907e533cc26\": container with ID starting with aed83ce8a5d1f5fe72f5ca4a1f68883b5170f515d0946f168e971907e533cc26 not found: ID does not exist"
Sep 18 18:09:37.267918 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:37.267877    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"]
Sep 18 18:09:37.280348 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:37.280323    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ef5655-66dc8c6477-ftq5r"]
Sep 18 18:09:38.045369 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:38.045340    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69f5d2a2-f501-4adc-9a78-88ab2449895b" path="/var/lib/kubelet/pods/69f5d2a2-f501-4adc-9a78-88ab2449895b/volumes"
Sep 18 18:09:38.045752 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:38.045736    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d009df22-fe7f-444f-bcee-4eba3eea7d8f" path="/var/lib/kubelet/pods/d009df22-fe7f-444f-bcee-4eba3eea7d8f/volumes"
Sep 18 18:09:49.408732 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.408700    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"]
Sep 18 18:09:49.409209 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.409105    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69f5d2a2-f501-4adc-9a78-88ab2449895b" containerName="istio-proxy"
Sep 18 18:09:49.409209 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.409115    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="69f5d2a2-f501-4adc-9a78-88ab2449895b" containerName="istio-proxy"
Sep 18 18:09:49.409209 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.409133    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a53b6b43-9321-47f6-a7a3-a54735d7d46d" containerName="maas-api"
Sep 18 18:09:49.409209 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.409138    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a53b6b43-9321-47f6-a7a3-a54735d7d46d" containerName="maas-api"
Sep 18 18:09:49.409209 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.409153    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d009df22-fe7f-444f-bcee-4eba3eea7d8f" containerName="maas-api"
Sep 18 18:09:49.409209 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.409159    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d009df22-fe7f-444f-bcee-4eba3eea7d8f" containerName="maas-api"
Sep 18 18:09:49.409456 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.409242    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d009df22-fe7f-444f-bcee-4eba3eea7d8f" containerName="maas-api"
Sep 18 18:09:49.409456 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.409251    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="69f5d2a2-f501-4adc-9a78-88ab2449895b" containerName="istio-proxy"
Sep 18 18:09:49.409456 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.409258    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a53b6b43-9321-47f6-a7a3-a54735d7d46d" containerName="maas-api"
Sep 18 18:09:49.409456 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.409266    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="69f5d2a2-f501-4adc-9a78-88ab2449895b" containerName="istio-proxy"
Sep 18 18:09:49.412367 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.412348    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.415441 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:09:49.415411    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"e2e-shared-a-w2-28ddec-openshift-default-dockercfg-srcz9\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-28ddec-openshift-default-dockercfg-srcz9\"" type="*v1.Secret"
Sep 18 18:09:49.415993 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:09:49.415968    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-gw-ca-root-cert\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" type="*v1.ConfigMap"
Sep 18 18:09:49.416170 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:09:49.416141    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" is forbidden: User \"system:node:ip-10-0-130-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-66.ec2.internal' and this object" podUID="4e322580-3bf4-44d3-9c10-54f664524069" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.447099 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.447069    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"]
Sep 18 18:09:49.504467 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.504430    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e322580-3bf4-44d3-9c10-54f664524069-istio-podinfo\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.504644 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.504475    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdv4n\" (UniqueName: \"kubernetes.io/projected/4e322580-3bf4-44d3-9c10-54f664524069-kube-api-access-mdv4n\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.504644 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.504556    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-credential-socket\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.504644 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.504583    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-istio-data\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.504823 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.504655    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-workload-socket\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.504823 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.504697    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e322580-3bf4-44d3-9c10-54f664524069-istio-token\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.504823 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.504732    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-workload-certs\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.504823 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.504761    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e322580-3bf4-44d3-9c10-54f664524069-istiod-ca-cert\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.504823 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.504799    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-istio-envoy\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.606049 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.606017    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e322580-3bf4-44d3-9c10-54f664524069-istio-podinfo\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.606049 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.606059    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdv4n\" (UniqueName: \"kubernetes.io/projected/4e322580-3bf4-44d3-9c10-54f664524069-kube-api-access-mdv4n\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.606324 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.606100    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-credential-socket\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.606324 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.606122    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-istio-data\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.606324 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.606242    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-workload-socket\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.606324 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.606308    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e322580-3bf4-44d3-9c10-54f664524069-istio-token\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.606561 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.606361    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-workload-certs\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.606561 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.606394    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e322580-3bf4-44d3-9c10-54f664524069-istiod-ca-cert\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.606561 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.606425    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-istio-envoy\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.606561 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.606499    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-istio-data\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.606783 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.606641    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-workload-socket\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.606783 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.606691    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-credential-socket\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.606783 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.606705    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-workload-certs\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.608475 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.608452    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-istio-envoy\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.608682 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.608662    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e322580-3bf4-44d3-9c10-54f664524069-istio-podinfo\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.631356 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.631330    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e322580-3bf4-44d3-9c10-54f664524069-istio-token\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:49.631693 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:49.631673    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdv4n\" (UniqueName: \"kubernetes.io/projected/4e322580-3bf4-44d3-9c10-54f664524069-kube-api-access-mdv4n\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:50.607421 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:09:50.607385    2568 configmap.go:193] Couldn't get configMap openshift-ingress/openshift-gw-ca-root-cert: failed to sync configmap cache: timed out waiting for the condition
Sep 18 18:09:50.607827 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:09:50.607497    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e322580-3bf4-44d3-9c10-54f664524069-istiod-ca-cert podName:4e322580-3bf4-44d3-9c10-54f664524069 nodeName:}" failed. No retries permitted until 2026-09-18 18:09:51.107480556 +0000 UTC m=+1195.185284433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "istiod-ca-cert" (UniqueName: "kubernetes.io/configmap/4e322580-3bf4-44d3-9c10-54f664524069-istiod-ca-cert") pod "e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" (UID: "4e322580-3bf4-44d3-9c10-54f664524069") : failed to sync configmap cache: timed out waiting for the condition
Sep 18 18:09:50.857791 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:50.857711    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-28ddec-openshift-default-dockercfg-srcz9\""
Sep 18 18:09:51.004783 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:51.004749    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 18:09:51.122559 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:51.122487    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e322580-3bf4-44d3-9c10-54f664524069-istiod-ca-cert\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:51.123125 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:51.123106    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e322580-3bf4-44d3-9c10-54f664524069-istiod-ca-cert\") pod \"e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") " pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:51.224640 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:51.224607    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:51.503962 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:51.503938    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"]
Sep 18 18:09:51.506084 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:09:51.506051    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e322580_3bf4_44d3_9c10_54f664524069.slice/crio-9428c6fbb3b02d364471f2d613eff369c9b3a0c287969b9af8077cde3d74335c WatchSource:0}: Error finding container 9428c6fbb3b02d364471f2d613eff369c9b3a0c287969b9af8077cde3d74335c: Status 404 returned error can't find the container with id 9428c6fbb3b02d364471f2d613eff369c9b3a0c287969b9af8077cde3d74335c
Sep 18 18:09:51.508307 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:51.508272    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:09:51.508390 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:51.508348    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:09:51.508390 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:51.508378    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:09:52.331775 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:52.331733    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" event={"ID":"4e322580-3bf4-44d3-9c10-54f664524069","Type":"ContainerStarted","Data":"a67725b77e91cca9b532ba79e308676c444c86d199596ef3154208cbcd083552"}
Sep 18 18:09:52.332167 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:52.331793    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" event={"ID":"4e322580-3bf4-44d3-9c10-54f664524069","Type":"ContainerStarted","Data":"9428c6fbb3b02d364471f2d613eff369c9b3a0c287969b9af8077cde3d74335c"}
Sep 18 18:09:52.377232 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:52.377161    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" podStartSLOduration=3.377143271 podStartE2EDuration="3.377143271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:09:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:09:52.375986106 +0000 UTC m=+1196.453790006" watchObservedRunningTime="2026-09-18 18:09:52.377143271 +0000 UTC m=+1196.454947169"
Sep 18 18:09:53.225600 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:53.225564    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:53.230695 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:53.230669    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:53.337574 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:53.337540    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:53.338517 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:53.338500    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:09:57.232457 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:57.232426    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:09:58.052938 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.052913    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 18:09:58.056739 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.056715    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 18:09:58.582960 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.582920    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"]
Sep 18 18:09:58.583596 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.583578    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69f5d2a2-f501-4adc-9a78-88ab2449895b" containerName="istio-proxy"
Sep 18 18:09:58.583596 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.583597    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="69f5d2a2-f501-4adc-9a78-88ab2449895b" containerName="istio-proxy"
Sep 18 18:09:58.588495 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.588474    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:09:58.591326 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.591302    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-28ddec-serving-cert\""
Sep 18 18:09:58.599022 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.598989    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"]
Sep 18 18:09:58.601588 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.601564    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/738736de-5f4e-468e-9971-60489e044a2f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:09:58.601719 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.601701    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/738736de-5f4e-468e-9971-60489e044a2f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:09:58.601796 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.601739    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/738736de-5f4e-468e-9971-60489e044a2f-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:09:58.601796 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.601778    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt965\" (UniqueName: \"kubernetes.io/projected/738736de-5f4e-468e-9971-60489e044a2f-kube-api-access-zt965\") pod \"maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:09:58.703060 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.703030    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt965\" (UniqueName: \"kubernetes.io/projected/738736de-5f4e-468e-9971-60489e044a2f-kube-api-access-zt965\") pod \"maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:09:58.703252 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.703096    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/738736de-5f4e-468e-9971-60489e044a2f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:09:58.703252 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.703242    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/738736de-5f4e-468e-9971-60489e044a2f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:09:58.703368 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.703268    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/738736de-5f4e-468e-9971-60489e044a2f-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:09:58.703428 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:09:58.703405    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-serving-cert: secret "maas-api-e2e-shared-a-w2-28ddec-serving-cert" not found
Sep 18 18:09:58.703472 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:09:58.703467    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/738736de-5f4e-468e-9971-60489e044a2f-maas-api-tls podName:738736de-5f4e-468e-9971-60489e044a2f nodeName:}" failed. No retries permitted until 2026-09-18 18:09:59.203448531 +0000 UTC m=+1203.281252416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/738736de-5f4e-468e-9971-60489e044a2f-maas-api-tls") pod "maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh" (UID: "738736de-5f4e-468e-9971-60489e044a2f") : secret "maas-api-e2e-shared-a-w2-28ddec-serving-cert" not found
Sep 18 18:09:58.703873 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.703847    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/738736de-5f4e-468e-9971-60489e044a2f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:09:58.705660 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.705636    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/738736de-5f4e-468e-9971-60489e044a2f-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:09:58.714106 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:58.714082    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt965\" (UniqueName: \"kubernetes.io/projected/738736de-5f4e-468e-9971-60489e044a2f-kube-api-access-zt965\") pod \"maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:09:59.208771 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:59.208734    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/738736de-5f4e-468e-9971-60489e044a2f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:09:59.211139 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:59.211117    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/738736de-5f4e-468e-9971-60489e044a2f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:09:59.501285 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:59.501202    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:09:59.787645 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:09:59.787619    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"]
Sep 18 18:09:59.789943 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:09:59.789916    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod738736de_5f4e_468e_9971_60489e044a2f.slice/crio-b4c60ce1f654fabc0ff84e3f69a22498fe47c34c5582217a07db9afc9c83d403 WatchSource:0}: Error finding container b4c60ce1f654fabc0ff84e3f69a22498fe47c34c5582217a07db9afc9c83d403: Status 404 returned error can't find the container with id b4c60ce1f654fabc0ff84e3f69a22498fe47c34c5582217a07db9afc9c83d403
Sep 18 18:10:00.383337 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:00.383303    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh" event={"ID":"738736de-5f4e-468e-9971-60489e044a2f","Type":"ContainerStarted","Data":"fd5655cb662738259e72881bceea5c5593371147c0f11f943a5c7331871095bb"}
Sep 18 18:10:00.383498 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:00.383353    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh" event={"ID":"738736de-5f4e-468e-9971-60489e044a2f","Type":"ContainerStarted","Data":"b4c60ce1f654fabc0ff84e3f69a22498fe47c34c5582217a07db9afc9c83d403"}
Sep 18 18:10:00.383498 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:00.383410    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:10:00.404407 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:00.404354    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh" podStartSLOduration=1.9255528100000001 podStartE2EDuration="2.404337909s" totalImagesPullingTime="478.785099ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:10:00.402491698 +0000 UTC m=+1204.480295599" watchObservedRunningTime="2026-09-18 18:10:00.404337909 +0000 UTC m=+1204.482141808"
Sep 18 18:10:06.393831 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:06.393804    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:10:20.501358 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.501327    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"]
Sep 18 18:10:20.505497 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.505475    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.509463 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.509440    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-2c937c-openshift-default-dockercfg-xwwlf\""
Sep 18 18:10:20.532606 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.532578    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"]
Sep 18 18:10:20.632608 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.632563    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-workload-certs\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.632801 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.632626    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-data\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.632801 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.632707    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-workload-socket\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.632801 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.632740    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2pc9\" (UniqueName: \"kubernetes.io/projected/b6d697e5-f817-481a-83b9-5b7dd3523cf8-kube-api-access-j2pc9\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.632801 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.632775    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-credential-socket\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.633009 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.632820    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-podinfo\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.633009 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.632909    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-envoy\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.633009 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.632980    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-token\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.633009 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.632999    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istiod-ca-cert\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.733865 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.733828    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-token\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.733865 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.733870    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istiod-ca-cert\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.734119 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.733916    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-workload-certs\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.734119 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.733955    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-data\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.734119 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.733981    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-workload-socket\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.734119 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.734003    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2pc9\" (UniqueName: \"kubernetes.io/projected/b6d697e5-f817-481a-83b9-5b7dd3523cf8-kube-api-access-j2pc9\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.734352 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.734215    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-credential-socket\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.734352 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.734253    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-podinfo\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.734352 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.734331    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-envoy\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.734519 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.734458    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-workload-certs\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.734576 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.734545    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-data\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.734625 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.734588    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-workload-socket\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.734625 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.734609    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-credential-socket\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.734766 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.734743    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istiod-ca-cert\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.736736 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.736715    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-envoy\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.736829 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.736757    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-podinfo\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.743376 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.743354    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2pc9\" (UniqueName: \"kubernetes.io/projected/b6d697e5-f817-481a-83b9-5b7dd3523cf8-kube-api-access-j2pc9\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.745937 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.745916    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-token\") pod \"e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") " pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:20.816757 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:20.816666    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:21.130314 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:21.130282    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"]
Sep 18 18:10:21.130440 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:10:21.130335    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d697e5_f817_481a_83b9_5b7dd3523cf8.slice/crio-2ceecb1ffdbf04327c775fdacf13e65891852d6f23faa78a29e66fea1f0e48a8 WatchSource:0}: Error finding container 2ceecb1ffdbf04327c775fdacf13e65891852d6f23faa78a29e66fea1f0e48a8: Status 404 returned error can't find the container with id 2ceecb1ffdbf04327c775fdacf13e65891852d6f23faa78a29e66fea1f0e48a8
Sep 18 18:10:21.132746 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:21.132714    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:10:21.132848 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:21.132777    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:10:21.132848 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:21.132805    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:10:21.508505 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:21.508468    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm" event={"ID":"b6d697e5-f817-481a-83b9-5b7dd3523cf8","Type":"ContainerStarted","Data":"d6b3fc7f93c4e2eedfcede4065e67c4eddc49753ca8475a3c1b6068436647316"}
Sep 18 18:10:21.508888 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:21.508524    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm" event={"ID":"b6d697e5-f817-481a-83b9-5b7dd3523cf8","Type":"ContainerStarted","Data":"2ceecb1ffdbf04327c775fdacf13e65891852d6f23faa78a29e66fea1f0e48a8"}
Sep 18 18:10:21.534350 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:21.534294    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm" podStartSLOduration=1.53427478 podStartE2EDuration="1.53427478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:10:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:10:21.533060056 +0000 UTC m=+1225.610863957" watchObservedRunningTime="2026-09-18 18:10:21.53427478 +0000 UTC m=+1225.612078680"
Sep 18 18:10:21.817473 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:21.817342    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:22.824322 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:22.824239    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:23.521250 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:23.521211    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:23.522306 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:23.522279    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:10:24.379342 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:24.379296    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 18:10:24.379739 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:24.379360    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" podUID="4e322580-3bf4-44d3-9c10-54f664524069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 18:10:26.539982 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:26.539948    2568 generic.go:356] "Generic (PLEG): container finished" podID="4e322580-3bf4-44d3-9c10-54f664524069" containerID="a67725b77e91cca9b532ba79e308676c444c86d199596ef3154208cbcd083552" exitCode=0
Sep 18 18:10:26.540568 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:26.540024    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" event={"ID":"4e322580-3bf4-44d3-9c10-54f664524069","Type":"ContainerDied","Data":"a67725b77e91cca9b532ba79e308676c444c86d199596ef3154208cbcd083552"}
Sep 18 18:10:26.540650 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:26.540602    2568 scope.go:117] "RemoveContainer" containerID="a67725b77e91cca9b532ba79e308676c444c86d199596ef3154208cbcd083552"
Sep 18 18:10:26.541834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:26.541805    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:10:26.541906 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:26.541871    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:10:26.541943 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:26.541904    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:10:27.547274 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:27.547242    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" event={"ID":"4e322580-3bf4-44d3-9c10-54f664524069","Type":"ContainerStarted","Data":"546d401f56050a801bcb0be5da3c9f43e06be418be304a70080972c09d303576"}
Sep 18 18:10:28.225466 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:28.225426    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:10:28.225642 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:28.225561    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 18 18:10:28.225642 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:28.225618    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" podUID="4e322580-3bf4-44d3-9c10-54f664524069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 18 18:10:29.225904 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:29.225869    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 18 18:10:29.226316 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:29.225937    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" podUID="4e322580-3bf4-44d3-9c10-54f664524069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 18 18:10:30.225785 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:30.225746    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 18 18:10:30.225989 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:30.225819    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" podUID="4e322580-3bf4-44d3-9c10-54f664524069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 18 18:10:31.225472 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:31.225432    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 18 18:10:31.225683 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:31.225512    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" podUID="4e322580-3bf4-44d3-9c10-54f664524069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 18 18:10:32.225193 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:32.225150    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 18 18:10:32.225664 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:32.225223    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" podUID="4e322580-3bf4-44d3-9c10-54f664524069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 18 18:10:33.229577 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:33.229544    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:10:33.230031 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:33.229969    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:10:33.230780 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:33.230760    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:10:36.033348 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:10:36.033314    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:11:02.131450 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:02.131368    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:11:31.830421 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:31.830384    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:11:44.717857 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.717819    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"]
Sep 18 18:11:44.721643 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.721622    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.724308 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.724288    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-609aba\"/\"distinct-w0-609aba-kserve-self-signed-certs\""
Sep 18 18:11:44.733603 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.733576    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"]
Sep 18 18:11:44.820491 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.820461    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwp2h\" (UniqueName: \"kubernetes.io/projected/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-kube-api-access-lwp2h\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.820651 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.820537    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-model-cache\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.820651 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.820571    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-home\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.820651 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.820607    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-tmp-dir\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.820651 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.820647    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-dshm\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.820808 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.820666    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-tls-certs\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.921239 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.921203    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwp2h\" (UniqueName: \"kubernetes.io/projected/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-kube-api-access-lwp2h\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.921419 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.921315    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-model-cache\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.921419 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.921353    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-home\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.921419 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.921402    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-tmp-dir\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.921598 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.921457    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-dshm\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.921598 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.921487    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-tls-certs\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.921814 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.921788    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-model-cache\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.921889 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.921812    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-tmp-dir\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.921889 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.921851    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-home\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.923771 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.923749    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-dshm\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.924052 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.924033    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-tls-certs\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:44.929465 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:44.929445    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwp2h\" (UniqueName: \"kubernetes.io/projected/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-kube-api-access-lwp2h\") pod \"distinct-w0-609aba-kserve-68bf856846-v4tr9\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") " pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:45.032035 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:45.031957    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:45.311080 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:45.311056    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"]
Sep 18 18:11:45.313158 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:11:45.313128    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c3d631c_0aac_4a24_8e64_0ef855ccb8a0.slice/crio-5ccb720777e207c5f33fa5988bececa8f086fc959ffe99462070fbaa4389138a WatchSource:0}: Error finding container 5ccb720777e207c5f33fa5988bececa8f086fc959ffe99462070fbaa4389138a: Status 404 returned error can't find the container with id 5ccb720777e207c5f33fa5988bececa8f086fc959ffe99462070fbaa4389138a
Sep 18 18:11:46.012276 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:46.012240    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9" event={"ID":"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0","Type":"ContainerStarted","Data":"6bd8d584a8dac7189588e9445934d74743e80e6e186781b7be037a366a25c212"}
Sep 18 18:11:46.012276 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:46.012284    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9" event={"ID":"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0","Type":"ContainerStarted","Data":"5ccb720777e207c5f33fa5988bececa8f086fc959ffe99462070fbaa4389138a"}
Sep 18 18:11:46.034614 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:46.034566    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9" podStartSLOduration=2.034551057 podStartE2EDuration="2.034551057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:11:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:11:46.031547481 +0000 UTC m=+1310.109351382" watchObservedRunningTime="2026-09-18 18:11:46.034551057 +0000 UTC m=+1310.112354955"
Sep 18 18:11:55.033134 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:55.033092    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:55.033629 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:55.033152    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:55.033856 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:55.033833    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:11:55.046084 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:55.046051    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:55.077761 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:55.077733    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:11:57.332546 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:11:57.332515    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"]
Sep 18 18:12:12.351820 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.351727    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9" podUID="6da81ff4-ca04-4289-bb42-fe0e5a9828b0" containerName="main" containerID="cri-o://354e0dc1576cf1e563cc14ca78d8b89ed2b6513ae1bfc26a220ca2eb3e449312" gracePeriod=45
Sep 18 18:12:12.605035 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.604972    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:12:12.708509 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.708472    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-model-cache\") pod \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") "
Sep 18 18:12:12.708698 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.708537    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-tls-certs\") pod \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") "
Sep 18 18:12:12.708698 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.708624    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rt2bq\" (UniqueName: \"kubernetes.io/projected/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-kube-api-access-rt2bq\") pod \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") "
Sep 18 18:12:12.708698 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.708671    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-home\") pod \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") "
Sep 18 18:12:12.708887 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.708710    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-tmp-dir\") pod \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") "
Sep 18 18:12:12.708887 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.708720    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-model-cache" (OuterVolumeSpecName: "model-cache") pod "6da81ff4-ca04-4289-bb42-fe0e5a9828b0" (UID: "6da81ff4-ca04-4289-bb42-fe0e5a9828b0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:12:12.708887 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.708815    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-dshm\") pod \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\" (UID: \"6da81ff4-ca04-4289-bb42-fe0e5a9828b0\") "
Sep 18 18:12:12.709034 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.708907    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-home" (OuterVolumeSpecName: "home") pod "6da81ff4-ca04-4289-bb42-fe0e5a9828b0" (UID: "6da81ff4-ca04-4289-bb42-fe0e5a9828b0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:12:12.709034 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.708926    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6da81ff4-ca04-4289-bb42-fe0e5a9828b0" (UID: "6da81ff4-ca04-4289-bb42-fe0e5a9828b0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:12:12.709250 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.709231    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-model-cache\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:12:12.709324 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.709255    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-home\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:12:12.709324 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.709269    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-tmp-dir\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:12:12.710937 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.710906    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6da81ff4-ca04-4289-bb42-fe0e5a9828b0" (UID: "6da81ff4-ca04-4289-bb42-fe0e5a9828b0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:12:12.711043 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.710940    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-dshm" (OuterVolumeSpecName: "dshm") pod "6da81ff4-ca04-4289-bb42-fe0e5a9828b0" (UID: "6da81ff4-ca04-4289-bb42-fe0e5a9828b0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:12:12.711043 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.711020    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-kube-api-access-rt2bq" (OuterVolumeSpecName: "kube-api-access-rt2bq") pod "6da81ff4-ca04-4289-bb42-fe0e5a9828b0" (UID: "6da81ff4-ca04-4289-bb42-fe0e5a9828b0"). InnerVolumeSpecName "kube-api-access-rt2bq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:12:12.810747 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.810705    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-dshm\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:12:12.810747 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.810739    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-tls-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:12:12.810747 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:12.810754    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt2bq\" (UniqueName: \"kubernetes.io/projected/6da81ff4-ca04-4289-bb42-fe0e5a9828b0-kube-api-access-rt2bq\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:12:13.169468 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:13.169430    2568 generic.go:356] "Generic (PLEG): container finished" podID="6da81ff4-ca04-4289-bb42-fe0e5a9828b0" containerID="354e0dc1576cf1e563cc14ca78d8b89ed2b6513ae1bfc26a220ca2eb3e449312" exitCode=0
Sep 18 18:12:13.169675 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:13.169475    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"
Sep 18 18:12:13.169675 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:13.169519    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9" event={"ID":"6da81ff4-ca04-4289-bb42-fe0e5a9828b0","Type":"ContainerDied","Data":"354e0dc1576cf1e563cc14ca78d8b89ed2b6513ae1bfc26a220ca2eb3e449312"}
Sep 18 18:12:13.169675 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:13.169559    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9" event={"ID":"6da81ff4-ca04-4289-bb42-fe0e5a9828b0","Type":"ContainerDied","Data":"62b2a0b1a36333a941b5c34e32dbab5e54985a1360bf0639b9447308dbd4c15c"}
Sep 18 18:12:13.169675 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:13.169577    2568 scope.go:117] "RemoveContainer" containerID="354e0dc1576cf1e563cc14ca78d8b89ed2b6513ae1bfc26a220ca2eb3e449312"
Sep 18 18:12:13.179573 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:13.179552    2568 scope.go:117] "RemoveContainer" containerID="354e0dc1576cf1e563cc14ca78d8b89ed2b6513ae1bfc26a220ca2eb3e449312"
Sep 18 18:12:13.179849 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:12:13.179832    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"354e0dc1576cf1e563cc14ca78d8b89ed2b6513ae1bfc26a220ca2eb3e449312\": container with ID starting with 354e0dc1576cf1e563cc14ca78d8b89ed2b6513ae1bfc26a220ca2eb3e449312 not found: ID does not exist" containerID="354e0dc1576cf1e563cc14ca78d8b89ed2b6513ae1bfc26a220ca2eb3e449312"
Sep 18 18:12:13.179889 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:13.179857    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"354e0dc1576cf1e563cc14ca78d8b89ed2b6513ae1bfc26a220ca2eb3e449312"} err="failed to get container status \"354e0dc1576cf1e563cc14ca78d8b89ed2b6513ae1bfc26a220ca2eb3e449312\": rpc error: code = NotFound desc = could not find container \"354e0dc1576cf1e563cc14ca78d8b89ed2b6513ae1bfc26a220ca2eb3e449312\": container with ID starting with 354e0dc1576cf1e563cc14ca78d8b89ed2b6513ae1bfc26a220ca2eb3e449312 not found: ID does not exist"
Sep 18 18:12:13.191559 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:13.191536    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"]
Sep 18 18:12:13.194735 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:13.194715    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-502939/test-model-502939-kserve-65848f44fc-59wz9"]
Sep 18 18:12:14.045825 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:14.045784    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6da81ff4-ca04-4289-bb42-fe0e5a9828b0" path="/var/lib/kubelet/pods/6da81ff4-ca04-4289-bb42-fe0e5a9828b0/volumes"
Sep 18 18:12:18.276287 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:18.276252    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:12:45.658462 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:45.658426    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"]
Sep 18 18:12:45.659392 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:45.659354    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk" podUID="f8923b82-671a-40a2-a5ea-86c8503e4683" containerName="payload-processing" containerID="cri-o://ccf17fbce2f9f9e0b4f1fe9501eeac59976a20bdfa2802b8ff62949d96db3116" gracePeriod=30
Sep 18 18:12:45.923074 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:45.923021    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"
Sep 18 18:12:45.941372 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:45.941346    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzjzb\" (UniqueName: \"kubernetes.io/projected/f8923b82-671a-40a2-a5ea-86c8503e4683-kube-api-access-qzjzb\") pod \"f8923b82-671a-40a2-a5ea-86c8503e4683\" (UID: \"f8923b82-671a-40a2-a5ea-86c8503e4683\") "
Sep 18 18:12:45.941482 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:45.941430    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8923b82-671a-40a2-a5ea-86c8503e4683-plugins-config-volume\") pod \"f8923b82-671a-40a2-a5ea-86c8503e4683\" (UID: \"f8923b82-671a-40a2-a5ea-86c8503e4683\") "
Sep 18 18:12:45.941836 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:45.941803    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8923b82-671a-40a2-a5ea-86c8503e4683-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f8923b82-671a-40a2-a5ea-86c8503e4683" (UID: "f8923b82-671a-40a2-a5ea-86c8503e4683"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:12:45.943884 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:45.943855    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8923b82-671a-40a2-a5ea-86c8503e4683-kube-api-access-qzjzb" (OuterVolumeSpecName: "kube-api-access-qzjzb") pod "f8923b82-671a-40a2-a5ea-86c8503e4683" (UID: "f8923b82-671a-40a2-a5ea-86c8503e4683"). InnerVolumeSpecName "kube-api-access-qzjzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:12:46.042333 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:46.042302    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8923b82-671a-40a2-a5ea-86c8503e4683-plugins-config-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:12:46.042333 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:46.042329    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzjzb\" (UniqueName: \"kubernetes.io/projected/f8923b82-671a-40a2-a5ea-86c8503e4683-kube-api-access-qzjzb\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:12:46.368728 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:46.368637    2568 generic.go:356] "Generic (PLEG): container finished" podID="f8923b82-671a-40a2-a5ea-86c8503e4683" containerID="ccf17fbce2f9f9e0b4f1fe9501eeac59976a20bdfa2802b8ff62949d96db3116" exitCode=0
Sep 18 18:12:46.368728 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:46.368688    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"
Sep 18 18:12:46.368926 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:46.368719    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk" event={"ID":"f8923b82-671a-40a2-a5ea-86c8503e4683","Type":"ContainerDied","Data":"ccf17fbce2f9f9e0b4f1fe9501eeac59976a20bdfa2802b8ff62949d96db3116"}
Sep 18 18:12:46.368926 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:46.368777    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk" event={"ID":"f8923b82-671a-40a2-a5ea-86c8503e4683","Type":"ContainerDied","Data":"f41dc94295c80971b6b4f571bb848b1d4a74a0e3ddd3da57ecef978e5e2d923f"}
Sep 18 18:12:46.368926 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:46.368799    2568 scope.go:117] "RemoveContainer" containerID="ccf17fbce2f9f9e0b4f1fe9501eeac59976a20bdfa2802b8ff62949d96db3116"
Sep 18 18:12:46.378230 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:46.378210    2568 scope.go:117] "RemoveContainer" containerID="ccf17fbce2f9f9e0b4f1fe9501eeac59976a20bdfa2802b8ff62949d96db3116"
Sep 18 18:12:46.378482 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:12:46.378462    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccf17fbce2f9f9e0b4f1fe9501eeac59976a20bdfa2802b8ff62949d96db3116\": container with ID starting with ccf17fbce2f9f9e0b4f1fe9501eeac59976a20bdfa2802b8ff62949d96db3116 not found: ID does not exist" containerID="ccf17fbce2f9f9e0b4f1fe9501eeac59976a20bdfa2802b8ff62949d96db3116"
Sep 18 18:12:46.378525 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:46.378489    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccf17fbce2f9f9e0b4f1fe9501eeac59976a20bdfa2802b8ff62949d96db3116"} err="failed to get container status \"ccf17fbce2f9f9e0b4f1fe9501eeac59976a20bdfa2802b8ff62949d96db3116\": rpc error: code = NotFound desc = could not find container \"ccf17fbce2f9f9e0b4f1fe9501eeac59976a20bdfa2802b8ff62949d96db3116\": container with ID starting with ccf17fbce2f9f9e0b4f1fe9501eeac59976a20bdfa2802b8ff62949d96db3116 not found: ID does not exist"
Sep 18 18:12:46.389880 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:46.389852    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"]
Sep 18 18:12:46.398101 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:46.398073    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ee3049-797c44b49f-5pbgk"]
Sep 18 18:12:48.046365 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:48.046333    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8923b82-671a-40a2-a5ea-86c8503e4683" path="/var/lib/kubelet/pods/f8923b82-671a-40a2-a5ea-86c8503e4683/volumes"
Sep 18 18:12:51.193250 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:51.193217    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"]
Sep 18 18:12:54.635513 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.635471    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"]
Sep 18 18:12:54.635989 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.635953    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8923b82-671a-40a2-a5ea-86c8503e4683" containerName="payload-processing"
Sep 18 18:12:54.635989 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.635972    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8923b82-671a-40a2-a5ea-86c8503e4683" containerName="payload-processing"
Sep 18 18:12:54.636075 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.636004    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6da81ff4-ca04-4289-bb42-fe0e5a9828b0" containerName="main"
Sep 18 18:12:54.636075 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.636013    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6da81ff4-ca04-4289-bb42-fe0e5a9828b0" containerName="main"
Sep 18 18:12:54.636159 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.636126    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6da81ff4-ca04-4289-bb42-fe0e5a9828b0" containerName="main"
Sep 18 18:12:54.636159 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.636139    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8923b82-671a-40a2-a5ea-86c8503e4683" containerName="payload-processing"
Sep 18 18:12:54.639489 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.639470    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.641802 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.641784    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-03e512\"/\"e2e-conflict-b-e4d74f58-kserve-self-signed-certs\""
Sep 18 18:12:54.651981 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.651958    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"]
Sep 18 18:12:54.721973 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.721943    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-dshm\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.722144 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.722012    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-home\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.722144 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.722062    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-model-cache\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.722291 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.722141    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmq7c\" (UniqueName: \"kubernetes.io/projected/7eaffe35-b703-4723-be05-fbbb3b80067c-kube-api-access-gmq7c\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.722291 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.722227    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eaffe35-b703-4723-be05-fbbb3b80067c-tls-certs\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.722378 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.722292    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-tmp-dir\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.741953 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.741920    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:12:54.822772 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.822740    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-tmp-dir\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.822941 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.822787    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-dshm\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.822941 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.822865    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-home\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.822941 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.822901    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-model-cache\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.823139 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.822973    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmq7c\" (UniqueName: \"kubernetes.io/projected/7eaffe35-b703-4723-be05-fbbb3b80067c-kube-api-access-gmq7c\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.823139 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.823003    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eaffe35-b703-4723-be05-fbbb3b80067c-tls-certs\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.823575 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.823313    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-home\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.823575 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.823533    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-model-cache\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.823899 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.823874    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-tmp-dir\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.825348 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.825325    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-dshm\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.825963 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.825929    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eaffe35-b703-4723-be05-fbbb3b80067c-tls-certs\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.833937 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.833913    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmq7c\" (UniqueName: \"kubernetes.io/projected/7eaffe35-b703-4723-be05-fbbb3b80067c-kube-api-access-gmq7c\") pod \"e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") " pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:54.948999 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:54.948967    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:12:55.229116 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:55.228875    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"]
Sep 18 18:12:55.232109 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:12:55.232071    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7eaffe35_b703_4723_be05_fbbb3b80067c.slice/crio-885838990e8e2f02ace61d6cacedd1d92a16e8e0eedc34c77680d531fc14d5da WatchSource:0}: Error finding container 885838990e8e2f02ace61d6cacedd1d92a16e8e0eedc34c77680d531fc14d5da: Status 404 returned error can't find the container with id 885838990e8e2f02ace61d6cacedd1d92a16e8e0eedc34c77680d531fc14d5da
Sep 18 18:12:55.426164 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:55.426132    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd" event={"ID":"7eaffe35-b703-4723-be05-fbbb3b80067c","Type":"ContainerStarted","Data":"19c0c042d26aa8c052784226662f1cc4452b8dc4376f258ff27b67800e911207"}
Sep 18 18:12:55.426164 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:55.426187    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd" event={"ID":"7eaffe35-b703-4723-be05-fbbb3b80067c","Type":"ContainerStarted","Data":"885838990e8e2f02ace61d6cacedd1d92a16e8e0eedc34c77680d531fc14d5da"}
Sep 18 18:12:55.452654 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:12:55.452601    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd" podStartSLOduration=1.452585069 podStartE2EDuration="1.452585069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:12:55.450528577 +0000 UTC m=+1379.528332477" watchObservedRunningTime="2026-09-18 18:12:55.452585069 +0000 UTC m=+1379.530388969"
Sep 18 18:13:02.025313 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:02.025265    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"]
Sep 18 18:13:02.025903 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:02.025500    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm" podUID="b6d697e5-f817-481a-83b9-5b7dd3523cf8" containerName="istio-proxy" containerID="cri-o://d6b3fc7f93c4e2eedfcede4065e67c4eddc49753ca8475a3c1b6068436647316" gracePeriod=30
Sep 18 18:13:04.949328 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:04.949288    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:13:04.949780 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:04.949343    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:13:04.967744 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:04.967716    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:13:05.493777 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:05.493747    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:13:06.212686 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.212640    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl" podUID="8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1" containerName="main" containerID="cri-o://722306262f91cac9024a31bff8fdb7abc7de623434e4c3b600256ef2fa419936" gracePeriod=45
Sep 18 18:13:06.463065 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.462999    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:13:06.490639 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.490607    2568 generic.go:356] "Generic (PLEG): container finished" podID="8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1" containerID="722306262f91cac9024a31bff8fdb7abc7de623434e4c3b600256ef2fa419936" exitCode=0
Sep 18 18:13:06.490639 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.490641    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"
Sep 18 18:13:06.490858 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.490690    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl" event={"ID":"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1","Type":"ContainerDied","Data":"722306262f91cac9024a31bff8fdb7abc7de623434e4c3b600256ef2fa419936"}
Sep 18 18:13:06.490858 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.490739    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl" event={"ID":"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1","Type":"ContainerDied","Data":"6c6273c6638b76558f8500ff80e83523dd2a3537bc5b87cd6204f824790745bd"}
Sep 18 18:13:06.490858 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.490764    2568 scope.go:117] "RemoveContainer" containerID="722306262f91cac9024a31bff8fdb7abc7de623434e4c3b600256ef2fa419936"
Sep 18 18:13:06.502347 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.501586    2568 scope.go:117] "RemoveContainer" containerID="722306262f91cac9024a31bff8fdb7abc7de623434e4c3b600256ef2fa419936"
Sep 18 18:13:06.502347 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:13:06.502061    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"722306262f91cac9024a31bff8fdb7abc7de623434e4c3b600256ef2fa419936\": container with ID starting with 722306262f91cac9024a31bff8fdb7abc7de623434e4c3b600256ef2fa419936 not found: ID does not exist" containerID="722306262f91cac9024a31bff8fdb7abc7de623434e4c3b600256ef2fa419936"
Sep 18 18:13:06.502347 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.502093    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"722306262f91cac9024a31bff8fdb7abc7de623434e4c3b600256ef2fa419936"} err="failed to get container status \"722306262f91cac9024a31bff8fdb7abc7de623434e4c3b600256ef2fa419936\": rpc error: code = NotFound desc = could not find container \"722306262f91cac9024a31bff8fdb7abc7de623434e4c3b600256ef2fa419936\": container with ID starting with 722306262f91cac9024a31bff8fdb7abc7de623434e4c3b600256ef2fa419936 not found: ID does not exist"
Sep 18 18:13:06.544625 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.544605    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-model-cache\") pod \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") "
Sep 18 18:13:06.544724 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.544669    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-home\") pod \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") "
Sep 18 18:13:06.544724 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.544691    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctkr6\" (UniqueName: \"kubernetes.io/projected/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-kube-api-access-ctkr6\") pod \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") "
Sep 18 18:13:06.544799 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.544749    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-tmp-dir\") pod \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") "
Sep 18 18:13:06.544799 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.544770    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-tls-certs\") pod \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") "
Sep 18 18:13:06.544873 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.544815    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-dshm\") pod \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\" (UID: \"8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1\") "
Sep 18 18:13:06.544916 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.544873    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-model-cache" (OuterVolumeSpecName: "model-cache") pod "8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1" (UID: "8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:06.544975 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.544956    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-home" (OuterVolumeSpecName: "home") pod "8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1" (UID: "8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:06.545029 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.545000    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1" (UID: "8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:06.545202 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.545159    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-tmp-dir\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:06.545202 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.545202    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-model-cache\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:06.545394 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.545213    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-home\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:06.546945 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.546924    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-dshm" (OuterVolumeSpecName: "dshm") pod "8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1" (UID: "8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:06.547020 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.546972    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-kube-api-access-ctkr6" (OuterVolumeSpecName: "kube-api-access-ctkr6") pod "8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1" (UID: "8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1"). InnerVolumeSpecName "kube-api-access-ctkr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:13:06.547060 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.547033    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1" (UID: "8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:13:06.645728 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.645693    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctkr6\" (UniqueName: \"kubernetes.io/projected/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-kube-api-access-ctkr6\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:06.645728 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.645722    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-tls-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:06.645728 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.645733    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1-dshm\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:06.815119 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.815084    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"]
Sep 18 18:13:06.817576 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:06.817555    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-ee3049/test-model-ee3049-kserve-ccff6c98f-rhjgl"]
Sep 18 18:13:07.279893 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.279865    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:13:07.352451 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.352421    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-workload-certs\") pod \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") "
Sep 18 18:13:07.352621 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.352465    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-envoy\") pod \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") "
Sep 18 18:13:07.352621 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.352492    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-credential-socket\") pod \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") "
Sep 18 18:13:07.352621 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.352512    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-data\") pod \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") "
Sep 18 18:13:07.352621 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.352530    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-podinfo\") pod \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") "
Sep 18 18:13:07.352621 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.352571    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istiod-ca-cert\") pod \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") "
Sep 18 18:13:07.352621 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.352614    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2pc9\" (UniqueName: \"kubernetes.io/projected/b6d697e5-f817-481a-83b9-5b7dd3523cf8-kube-api-access-j2pc9\") pod \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") "
Sep 18 18:13:07.352928 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.352636    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-token\") pod \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") "
Sep 18 18:13:07.352928 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.352708    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-workload-socket\") pod \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\" (UID: \"b6d697e5-f817-481a-83b9-5b7dd3523cf8\") "
Sep 18 18:13:07.352928 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.352711    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b6d697e5-f817-481a-83b9-5b7dd3523cf8" (UID: "b6d697e5-f817-481a-83b9-5b7dd3523cf8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:07.352928 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.352830    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b6d697e5-f817-481a-83b9-5b7dd3523cf8" (UID: "b6d697e5-f817-481a-83b9-5b7dd3523cf8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:07.352928 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.352848    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-data" (OuterVolumeSpecName: "istio-data") pod "b6d697e5-f817-481a-83b9-5b7dd3523cf8" (UID: "b6d697e5-f817-481a-83b9-5b7dd3523cf8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:07.353209 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.353163    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:07.353290 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.353206    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:07.353290 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.353222    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:07.353391 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.353316    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b6d697e5-f817-481a-83b9-5b7dd3523cf8" (UID: "b6d697e5-f817-481a-83b9-5b7dd3523cf8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:13:07.353391 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.353374    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b6d697e5-f817-481a-83b9-5b7dd3523cf8" (UID: "b6d697e5-f817-481a-83b9-5b7dd3523cf8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:07.354997 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.354977    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b6d697e5-f817-481a-83b9-5b7dd3523cf8" (UID: "b6d697e5-f817-481a-83b9-5b7dd3523cf8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:07.355117 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.355029    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6d697e5-f817-481a-83b9-5b7dd3523cf8-kube-api-access-j2pc9" (OuterVolumeSpecName: "kube-api-access-j2pc9") pod "b6d697e5-f817-481a-83b9-5b7dd3523cf8" (UID: "b6d697e5-f817-481a-83b9-5b7dd3523cf8"). InnerVolumeSpecName "kube-api-access-j2pc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:13:07.355117 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.355089    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b6d697e5-f817-481a-83b9-5b7dd3523cf8" (UID: "b6d697e5-f817-481a-83b9-5b7dd3523cf8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:13:07.355249 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.355115    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-token" (OuterVolumeSpecName: "istio-token") pod "b6d697e5-f817-481a-83b9-5b7dd3523cf8" (UID: "b6d697e5-f817-481a-83b9-5b7dd3523cf8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:13:07.454136 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.454110    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:07.454136 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.454134    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:07.454349 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.454143    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:07.454349 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.454152    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2pc9\" (UniqueName: \"kubernetes.io/projected/b6d697e5-f817-481a-83b9-5b7dd3523cf8-kube-api-access-j2pc9\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:07.454349 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.454161    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6d697e5-f817-481a-83b9-5b7dd3523cf8-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:07.454349 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.454190    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d697e5-f817-481a-83b9-5b7dd3523cf8-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:07.497211 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.497161    2568 generic.go:356] "Generic (PLEG): container finished" podID="b6d697e5-f817-481a-83b9-5b7dd3523cf8" containerID="d6b3fc7f93c4e2eedfcede4065e67c4eddc49753ca8475a3c1b6068436647316" exitCode=0
Sep 18 18:13:07.497368 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.497219    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"
Sep 18 18:13:07.497368 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.497279    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm" event={"ID":"b6d697e5-f817-481a-83b9-5b7dd3523cf8","Type":"ContainerDied","Data":"d6b3fc7f93c4e2eedfcede4065e67c4eddc49753ca8475a3c1b6068436647316"}
Sep 18 18:13:07.497368 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.497361    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm" event={"ID":"b6d697e5-f817-481a-83b9-5b7dd3523cf8","Type":"ContainerDied","Data":"2ceecb1ffdbf04327c775fdacf13e65891852d6f23faa78a29e66fea1f0e48a8"}
Sep 18 18:13:07.497533 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.497393    2568 scope.go:117] "RemoveContainer" containerID="d6b3fc7f93c4e2eedfcede4065e67c4eddc49753ca8475a3c1b6068436647316"
Sep 18 18:13:07.506803 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.506786    2568 scope.go:117] "RemoveContainer" containerID="d6b3fc7f93c4e2eedfcede4065e67c4eddc49753ca8475a3c1b6068436647316"
Sep 18 18:13:07.507086 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:13:07.507067    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6b3fc7f93c4e2eedfcede4065e67c4eddc49753ca8475a3c1b6068436647316\": container with ID starting with d6b3fc7f93c4e2eedfcede4065e67c4eddc49753ca8475a3c1b6068436647316 not found: ID does not exist" containerID="d6b3fc7f93c4e2eedfcede4065e67c4eddc49753ca8475a3c1b6068436647316"
Sep 18 18:13:07.507135 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.507095    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6b3fc7f93c4e2eedfcede4065e67c4eddc49753ca8475a3c1b6068436647316"} err="failed to get container status \"d6b3fc7f93c4e2eedfcede4065e67c4eddc49753ca8475a3c1b6068436647316\": rpc error: code = NotFound desc = could not find container \"d6b3fc7f93c4e2eedfcede4065e67c4eddc49753ca8475a3c1b6068436647316\": container with ID starting with d6b3fc7f93c4e2eedfcede4065e67c4eddc49753ca8475a3c1b6068436647316 not found: ID does not exist"
Sep 18 18:13:07.519696 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.519670    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"]
Sep 18 18:13:07.524574 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:07.524553    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-2c937c-openshift-default-5d455f6c54-qvwtm"]
Sep 18 18:13:08.046924 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:08.046891    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1" path="/var/lib/kubelet/pods/8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1/volumes"
Sep 18 18:13:08.047359 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:08.047345    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6d697e5-f817-481a-83b9-5b7dd3523cf8" path="/var/lib/kubelet/pods/b6d697e5-f817-481a-83b9-5b7dd3523cf8/volumes"
Sep 18 18:13:14.932458 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:14.932422    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:13:21.393148 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.393113    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"]
Sep 18 18:13:21.393640 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.393624    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1" containerName="main"
Sep 18 18:13:21.393701 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.393643    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1" containerName="main"
Sep 18 18:13:21.393701 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.393652    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6d697e5-f817-481a-83b9-5b7dd3523cf8" containerName="istio-proxy"
Sep 18 18:13:21.393701 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.393657    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6d697e5-f817-481a-83b9-5b7dd3523cf8" containerName="istio-proxy"
Sep 18 18:13:21.393802 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.393747    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f0fcaaf-adff-4b5a-a0ad-154ebc8ffee1" containerName="main"
Sep 18 18:13:21.393802 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.393759    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6d697e5-f817-481a-83b9-5b7dd3523cf8" containerName="istio-proxy"
Sep 18 18:13:21.396792 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.396777    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.401647 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.401626    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f7e72d9f\"/\"kube-root-ca.crt\""
Sep 18 18:13:21.402542 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.402513    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f7e72d9f\"/\"default-dockercfg-dlwhd\""
Sep 18 18:13:21.402542 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.402532    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f7e72d9f\"/\"openshift-service-ca.crt\""
Sep 18 18:13:21.402711 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.402535    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f7e72d9f\"/\"e2e-lifecycle-model-f7e72d9f-kserve-self-signed-certs\""
Sep 18 18:13:21.437747 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.437718    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"]
Sep 18 18:13:21.494284 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.494253    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed102bd5-e4f3-4963-8a25-f9996157762e-tls-certs\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.494469 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.494296    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-dshm\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.494469 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.494327    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-tmp-dir\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.494469 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.494409    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-home\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.494634 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.494506    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ggxb\" (UniqueName: \"kubernetes.io/projected/ed102bd5-e4f3-4963-8a25-f9996157762e-kube-api-access-7ggxb\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.494634 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.494527    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-model-cache\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.595880 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.595847    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed102bd5-e4f3-4963-8a25-f9996157762e-tls-certs\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.596365 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.595890    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-dshm\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.596365 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.595927    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-tmp-dir\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.596365 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.595965    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-home\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.596365 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.596060    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ggxb\" (UniqueName: \"kubernetes.io/projected/ed102bd5-e4f3-4963-8a25-f9996157762e-kube-api-access-7ggxb\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.596365 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.596090    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-model-cache\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.596690 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.596436    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-tmp-dir\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.596690 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.596466    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-home\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.596690 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.596484    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-model-cache\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.598306 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.598283    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-dshm\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.598570 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.598553    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed102bd5-e4f3-4963-8a25-f9996157762e-tls-certs\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.609343 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.609320    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ggxb\" (UniqueName: \"kubernetes.io/projected/ed102bd5-e4f3-4963-8a25-f9996157762e-kube-api-access-7ggxb\") pod \"e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") " pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.706646 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.706614    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:21.987777 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:21.987748    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"]
Sep 18 18:13:21.989450 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:13:21.989424    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded102bd5_e4f3_4963_8a25_f9996157762e.slice/crio-47f768af58b6cfccaf79b011154ec798a3ba7ba14d3e3cb25f375e34e58b2ee2 WatchSource:0}: Error finding container 47f768af58b6cfccaf79b011154ec798a3ba7ba14d3e3cb25f375e34e58b2ee2: Status 404 returned error can't find the container with id 47f768af58b6cfccaf79b011154ec798a3ba7ba14d3e3cb25f375e34e58b2ee2
Sep 18 18:13:22.584798 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:22.584759    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl" event={"ID":"ed102bd5-e4f3-4963-8a25-f9996157762e","Type":"ContainerStarted","Data":"8f17dbdbb9441b8c2d8f3926dc14807aa9ea3905d864ad0a8a9969fca518debb"}
Sep 18 18:13:22.585324 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:22.584813    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl" event={"ID":"ed102bd5-e4f3-4963-8a25-f9996157762e","Type":"ContainerStarted","Data":"47f768af58b6cfccaf79b011154ec798a3ba7ba14d3e3cb25f375e34e58b2ee2"}
Sep 18 18:13:22.613918 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:22.613854    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl" podStartSLOduration=1.613832488 podStartE2EDuration="1.613832488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:13:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:13:22.603595297 +0000 UTC m=+1406.681399196" watchObservedRunningTime="2026-09-18 18:13:22.613832488 +0000 UTC m=+1406.691636389"
Sep 18 18:13:25.020675 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:25.020637    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"]
Sep 18 18:13:25.046221 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:25.046191    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"]
Sep 18 18:13:29.777058 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.777023    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"]
Sep 18 18:13:29.781569 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.781545    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.783957 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.783930    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-53cde3-openshift-default-dockercfg-gwf5f\""
Sep 18 18:13:29.794498 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.794473    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"]
Sep 18 18:13:29.881542 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.881508    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-envoy\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.881542 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.881551    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-workload-socket\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.881812 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.881612    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-data\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.881812 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.881714    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-workload-certs\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.881812 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.881746    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e083287-05ce-4370-b6d9-eb248cb8af8f-istiod-ca-cert\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.881812 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.881794    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8gnw\" (UniqueName: \"kubernetes.io/projected/4e083287-05ce-4370-b6d9-eb248cb8af8f-kube-api-access-w8gnw\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.881972 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.881826    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-credential-socket\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.881972 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.881923    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-podinfo\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.881972 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.881966    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-token\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.982503 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.982469    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-workload-certs\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.982503 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.982508    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e083287-05ce-4370-b6d9-eb248cb8af8f-istiod-ca-cert\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.982717 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.982534    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8gnw\" (UniqueName: \"kubernetes.io/projected/4e083287-05ce-4370-b6d9-eb248cb8af8f-kube-api-access-w8gnw\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.982717 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.982564    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-credential-socket\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.982717 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.982619    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-podinfo\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.982851 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.982824    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-token\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.982908 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.982885    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-envoy\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.982960 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.982914    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-workload-socket\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.982960 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.982915    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-workload-certs\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.983064 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.982951    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-credential-socket\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.983064 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.982977    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-data\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.983248 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.983222    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-workload-socket\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.983371 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.983337    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e083287-05ce-4370-b6d9-eb248cb8af8f-istiod-ca-cert\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.983503 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.983381    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-data\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.985306 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.985287    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-podinfo\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.985415 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.985302    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-envoy\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.990739 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.990716    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-token\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:29.991133 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:29.991112    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8gnw\" (UniqueName: \"kubernetes.io/projected/4e083287-05ce-4370-b6d9-eb248cb8af8f-kube-api-access-w8gnw\") pod \"e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") " pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:30.093915 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:30.093838    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:30.374883 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:30.374860    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"]
Sep 18 18:13:30.376974 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:13:30.376945    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e083287_05ce_4370_b6d9_eb248cb8af8f.slice/crio-6583335451a309a8fd64d0f9ae1ce475846669cd1b3a8ddc61fee52ade95897d WatchSource:0}: Error finding container 6583335451a309a8fd64d0f9ae1ce475846669cd1b3a8ddc61fee52ade95897d: Status 404 returned error can't find the container with id 6583335451a309a8fd64d0f9ae1ce475846669cd1b3a8ddc61fee52ade95897d
Sep 18 18:13:30.379687 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:30.379653    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:13:30.379773 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:30.379717    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:13:30.379773 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:30.379745    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:13:30.632590 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:30.632499    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm" event={"ID":"4e083287-05ce-4370-b6d9-eb248cb8af8f","Type":"ContainerStarted","Data":"2b818ac88151ba56bcb8a04f67de0a28d5887507426ae4602b36ee8e08edaa74"}
Sep 18 18:13:30.632590 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:30.632560    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm" event={"ID":"4e083287-05ce-4370-b6d9-eb248cb8af8f","Type":"ContainerStarted","Data":"6583335451a309a8fd64d0f9ae1ce475846669cd1b3a8ddc61fee52ade95897d"}
Sep 18 18:13:30.656081 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:30.656027    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm" podStartSLOduration=1.656010336 podStartE2EDuration="1.656010336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:13:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:13:30.654389711 +0000 UTC m=+1414.732193623" watchObservedRunningTime="2026-09-18 18:13:30.656010336 +0000 UTC m=+1414.733814234"
Sep 18 18:13:31.095014 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:31.094980    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:31.100276 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:31.100249    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:31.639291 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:31.639252    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:31.640529 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:31.640502    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:13:31.641116 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:31.641092    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"]
Sep 18 18:13:31.641333 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:31.641312    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2" podUID="f1fec7ec-ea60-4ae4-8ac6-4748b259986b" containerName="maas-api" containerID="cri-o://f410f25f04836e117f9a9ae5a9305bc1deb001f497d40be2db65b9527d0cb62b" gracePeriod=30
Sep 18 18:13:31.707668 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:31.707635    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:31.707824 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:31.707690    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:31.722678 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:31.722654    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:31.899343 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:31.899281    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:13:32.002513 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.002483    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-metrics-certs\") pod \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") "
Sep 18 18:13:32.002670 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.002545    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8tst\" (UniqueName: \"kubernetes.io/projected/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-kube-api-access-w8tst\") pod \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") "
Sep 18 18:13:32.002670 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.002625    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-maas-api-tls\") pod \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") "
Sep 18 18:13:32.002670 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.002664    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-openshift-service-ca\") pod \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\" (UID: \"f1fec7ec-ea60-4ae4-8ac6-4748b259986b\") "
Sep 18 18:13:32.003076 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.003052    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f1fec7ec-ea60-4ae4-8ac6-4748b259986b" (UID: "f1fec7ec-ea60-4ae4-8ac6-4748b259986b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:13:32.004705 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.004674    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f1fec7ec-ea60-4ae4-8ac6-4748b259986b" (UID: "f1fec7ec-ea60-4ae4-8ac6-4748b259986b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:13:32.004805 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.004704    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f1fec7ec-ea60-4ae4-8ac6-4748b259986b" (UID: "f1fec7ec-ea60-4ae4-8ac6-4748b259986b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:13:32.004849 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.004808    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-kube-api-access-w8tst" (OuterVolumeSpecName: "kube-api-access-w8tst") pod "f1fec7ec-ea60-4ae4-8ac6-4748b259986b" (UID: "f1fec7ec-ea60-4ae4-8ac6-4748b259986b"). InnerVolumeSpecName "kube-api-access-w8tst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:13:32.104477 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.104445    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-metrics-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:32.104477 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.104473    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8tst\" (UniqueName: \"kubernetes.io/projected/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-kube-api-access-w8tst\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:32.104836 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.104485    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:32.104836 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.104499    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1fec7ec-ea60-4ae4-8ac6-4748b259986b-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:32.645449 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.645412    2568 generic.go:356] "Generic (PLEG): container finished" podID="f1fec7ec-ea60-4ae4-8ac6-4748b259986b" containerID="f410f25f04836e117f9a9ae5a9305bc1deb001f497d40be2db65b9527d0cb62b" exitCode=0
Sep 18 18:13:32.645657 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.645465    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"
Sep 18 18:13:32.645657 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.645497    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2" event={"ID":"f1fec7ec-ea60-4ae4-8ac6-4748b259986b","Type":"ContainerDied","Data":"f410f25f04836e117f9a9ae5a9305bc1deb001f497d40be2db65b9527d0cb62b"}
Sep 18 18:13:32.645657 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.645550    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2" event={"ID":"f1fec7ec-ea60-4ae4-8ac6-4748b259986b","Type":"ContainerDied","Data":"1a73b9eef51168290c9f50950df62bbcda9678d5215b6e6e1c20dd80952e59c1"}
Sep 18 18:13:32.645657 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.645579    2568 scope.go:117] "RemoveContainer" containerID="f410f25f04836e117f9a9ae5a9305bc1deb001f497d40be2db65b9527d0cb62b"
Sep 18 18:13:32.655533 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.655516    2568 scope.go:117] "RemoveContainer" containerID="f410f25f04836e117f9a9ae5a9305bc1deb001f497d40be2db65b9527d0cb62b"
Sep 18 18:13:32.655829 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:13:32.655812    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f410f25f04836e117f9a9ae5a9305bc1deb001f497d40be2db65b9527d0cb62b\": container with ID starting with f410f25f04836e117f9a9ae5a9305bc1deb001f497d40be2db65b9527d0cb62b not found: ID does not exist" containerID="f410f25f04836e117f9a9ae5a9305bc1deb001f497d40be2db65b9527d0cb62b"
Sep 18 18:13:32.655904 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.655837    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f410f25f04836e117f9a9ae5a9305bc1deb001f497d40be2db65b9527d0cb62b"} err="failed to get container status \"f410f25f04836e117f9a9ae5a9305bc1deb001f497d40be2db65b9527d0cb62b\": rpc error: code = NotFound desc = could not find container \"f410f25f04836e117f9a9ae5a9305bc1deb001f497d40be2db65b9527d0cb62b\": container with ID starting with f410f25f04836e117f9a9ae5a9305bc1deb001f497d40be2db65b9527d0cb62b not found: ID does not exist"
Sep 18 18:13:32.659055 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.659033    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:13:32.669859 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.669835    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"]
Sep 18 18:13:32.675328 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:32.675304    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-609aba-59978f9d4b-qkqf2"]
Sep 18 18:13:34.046154 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:34.046118    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1fec7ec-ea60-4ae4-8ac6-4748b259986b" path="/var/lib/kubelet/pods/f1fec7ec-ea60-4ae4-8ac6-4748b259986b/volumes"
Sep 18 18:13:36.248368 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:36.248338    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:13:40.040528 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.040485    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9" podUID="4c3d631c-0aac-4a24-8e64-0ef855ccb8a0" containerName="main" containerID="cri-o://6bd8d584a8dac7189588e9445934d74743e80e6e186781b7be037a366a25c212" gracePeriod=45
Sep 18 18:13:40.065781 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.065726    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv" podUID="55e9632e-d0c7-456d-a2f8-a3d998e8cd29" containerName="main" containerID="cri-o://19011cc33a3095e322a0eb3f4366e5547f1956ebd9242a32775c07b3ef379e2a" gracePeriod=45
Sep 18 18:13:40.349076 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.349043    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:13:40.352327 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.352304    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:13:40.390896 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.390865    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-home\") pod \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") "
Sep 18 18:13:40.391232 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.391204    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-home" (OuterVolumeSpecName: "home") pod "4c3d631c-0aac-4a24-8e64-0ef855ccb8a0" (UID: "4c3d631c-0aac-4a24-8e64-0ef855ccb8a0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:40.391684 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.391225    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-tls-certs\") pod \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") "
Sep 18 18:13:40.391792 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.391732    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-tmp-dir\") pod \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") "
Sep 18 18:13:40.391792 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.391778    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-model-cache\") pod \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") "
Sep 18 18:13:40.391912 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.391820    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwp2h\" (UniqueName: \"kubernetes.io/projected/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-kube-api-access-lwp2h\") pod \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") "
Sep 18 18:13:40.392791 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.392126    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-dshm\") pod \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\" (UID: \"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0\") "
Sep 18 18:13:40.392791 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.392656    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-model-cache" (OuterVolumeSpecName: "model-cache") pod "4c3d631c-0aac-4a24-8e64-0ef855ccb8a0" (UID: "4c3d631c-0aac-4a24-8e64-0ef855ccb8a0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:40.392791 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.392709    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4c3d631c-0aac-4a24-8e64-0ef855ccb8a0" (UID: "4c3d631c-0aac-4a24-8e64-0ef855ccb8a0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:40.392791 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.392745    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-home\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:40.394112 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.394090    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4c3d631c-0aac-4a24-8e64-0ef855ccb8a0" (UID: "4c3d631c-0aac-4a24-8e64-0ef855ccb8a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:13:40.394848 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.394815    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-kube-api-access-lwp2h" (OuterVolumeSpecName: "kube-api-access-lwp2h") pod "4c3d631c-0aac-4a24-8e64-0ef855ccb8a0" (UID: "4c3d631c-0aac-4a24-8e64-0ef855ccb8a0"). InnerVolumeSpecName "kube-api-access-lwp2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:13:40.395744 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.395718    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-dshm" (OuterVolumeSpecName: "dshm") pod "4c3d631c-0aac-4a24-8e64-0ef855ccb8a0" (UID: "4c3d631c-0aac-4a24-8e64-0ef855ccb8a0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:40.493459 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.493379    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-tls-certs\") pod \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") "
Sep 18 18:13:40.493638 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.493497    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-home\") pod \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") "
Sep 18 18:13:40.493638 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.493523    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-dshm\") pod \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") "
Sep 18 18:13:40.493638 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.493550    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l5bh\" (UniqueName: \"kubernetes.io/projected/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-kube-api-access-5l5bh\") pod \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") "
Sep 18 18:13:40.493638 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.493569    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-tmp-dir\") pod \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") "
Sep 18 18:13:40.493638 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.493612    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-model-cache\") pod \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\" (UID: \"55e9632e-d0c7-456d-a2f8-a3d998e8cd29\") "
Sep 18 18:13:40.493920 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.493845    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-tls-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:40.493920 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.493872    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-tmp-dir\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:40.493920 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.493886    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-model-cache\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:40.493920 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.493899    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwp2h\" (UniqueName: \"kubernetes.io/projected/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-kube-api-access-lwp2h\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:40.493920 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.493910    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0-dshm\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:40.493920 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.493838    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-home" (OuterVolumeSpecName: "home") pod "55e9632e-d0c7-456d-a2f8-a3d998e8cd29" (UID: "55e9632e-d0c7-456d-a2f8-a3d998e8cd29"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:40.494203 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.493881    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "55e9632e-d0c7-456d-a2f8-a3d998e8cd29" (UID: "55e9632e-d0c7-456d-a2f8-a3d998e8cd29"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:40.494203 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.493956    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-model-cache" (OuterVolumeSpecName: "model-cache") pod "55e9632e-d0c7-456d-a2f8-a3d998e8cd29" (UID: "55e9632e-d0c7-456d-a2f8-a3d998e8cd29"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:40.495721 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.495693    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "55e9632e-d0c7-456d-a2f8-a3d998e8cd29" (UID: "55e9632e-d0c7-456d-a2f8-a3d998e8cd29"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:13:40.495821 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.495721    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-dshm" (OuterVolumeSpecName: "dshm") pod "55e9632e-d0c7-456d-a2f8-a3d998e8cd29" (UID: "55e9632e-d0c7-456d-a2f8-a3d998e8cd29"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:13:40.495821 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.495739    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-kube-api-access-5l5bh" (OuterVolumeSpecName: "kube-api-access-5l5bh") pod "55e9632e-d0c7-456d-a2f8-a3d998e8cd29" (UID: "55e9632e-d0c7-456d-a2f8-a3d998e8cd29"). InnerVolumeSpecName "kube-api-access-5l5bh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:13:40.594567 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.594485    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-home\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:40.594567 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.594513    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-dshm\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:40.594567 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.594523    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l5bh\" (UniqueName: \"kubernetes.io/projected/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-kube-api-access-5l5bh\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:40.594567 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.594533    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-tmp-dir\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:40.594567 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.594542    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-model-cache\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:40.594567 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.594549    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55e9632e-d0c7-456d-a2f8-a3d998e8cd29-tls-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:13:40.696645 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.696609    2568 generic.go:356] "Generic (PLEG): container finished" podID="55e9632e-d0c7-456d-a2f8-a3d998e8cd29" containerID="19011cc33a3095e322a0eb3f4366e5547f1956ebd9242a32775c07b3ef379e2a" exitCode=0
Sep 18 18:13:40.696834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.696649    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"
Sep 18 18:13:40.696834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.696692    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv" event={"ID":"55e9632e-d0c7-456d-a2f8-a3d998e8cd29","Type":"ContainerDied","Data":"19011cc33a3095e322a0eb3f4366e5547f1956ebd9242a32775c07b3ef379e2a"}
Sep 18 18:13:40.696834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.696732    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv" event={"ID":"55e9632e-d0c7-456d-a2f8-a3d998e8cd29","Type":"ContainerDied","Data":"69f305cc0825d831be67f6e79bdc6cd2d6e5e791231dbc60dc9f13bbbe9bcd0d"}
Sep 18 18:13:40.696834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.696751    2568 scope.go:117] "RemoveContainer" containerID="19011cc33a3095e322a0eb3f4366e5547f1956ebd9242a32775c07b3ef379e2a"
Sep 18 18:13:40.698351 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.698325    2568 generic.go:356] "Generic (PLEG): container finished" podID="4c3d631c-0aac-4a24-8e64-0ef855ccb8a0" containerID="6bd8d584a8dac7189588e9445934d74743e80e6e186781b7be037a366a25c212" exitCode=0
Sep 18 18:13:40.698448 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.698370    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"
Sep 18 18:13:40.698448 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.698387    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9" event={"ID":"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0","Type":"ContainerDied","Data":"6bd8d584a8dac7189588e9445934d74743e80e6e186781b7be037a366a25c212"}
Sep 18 18:13:40.698448 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.698423    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9" event={"ID":"4c3d631c-0aac-4a24-8e64-0ef855ccb8a0","Type":"ContainerDied","Data":"5ccb720777e207c5f33fa5988bececa8f086fc959ffe99462070fbaa4389138a"}
Sep 18 18:13:40.705775 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.705756    2568 scope.go:117] "RemoveContainer" containerID="19011cc33a3095e322a0eb3f4366e5547f1956ebd9242a32775c07b3ef379e2a"
Sep 18 18:13:40.706057 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:13:40.706035    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19011cc33a3095e322a0eb3f4366e5547f1956ebd9242a32775c07b3ef379e2a\": container with ID starting with 19011cc33a3095e322a0eb3f4366e5547f1956ebd9242a32775c07b3ef379e2a not found: ID does not exist" containerID="19011cc33a3095e322a0eb3f4366e5547f1956ebd9242a32775c07b3ef379e2a"
Sep 18 18:13:40.706126 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.706070    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19011cc33a3095e322a0eb3f4366e5547f1956ebd9242a32775c07b3ef379e2a"} err="failed to get container status \"19011cc33a3095e322a0eb3f4366e5547f1956ebd9242a32775c07b3ef379e2a\": rpc error: code = NotFound desc = could not find container \"19011cc33a3095e322a0eb3f4366e5547f1956ebd9242a32775c07b3ef379e2a\": container with ID starting with 19011cc33a3095e322a0eb3f4366e5547f1956ebd9242a32775c07b3ef379e2a not found: ID does not exist"
Sep 18 18:13:40.706126 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.706093    2568 scope.go:117] "RemoveContainer" containerID="6bd8d584a8dac7189588e9445934d74743e80e6e186781b7be037a366a25c212"
Sep 18 18:13:40.714989 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.714972    2568 scope.go:117] "RemoveContainer" containerID="6bd8d584a8dac7189588e9445934d74743e80e6e186781b7be037a366a25c212"
Sep 18 18:13:40.715259 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:13:40.715242    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bd8d584a8dac7189588e9445934d74743e80e6e186781b7be037a366a25c212\": container with ID starting with 6bd8d584a8dac7189588e9445934d74743e80e6e186781b7be037a366a25c212 not found: ID does not exist" containerID="6bd8d584a8dac7189588e9445934d74743e80e6e186781b7be037a366a25c212"
Sep 18 18:13:40.715335 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.715266    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bd8d584a8dac7189588e9445934d74743e80e6e186781b7be037a366a25c212"} err="failed to get container status \"6bd8d584a8dac7189588e9445934d74743e80e6e186781b7be037a366a25c212\": rpc error: code = NotFound desc = could not find container \"6bd8d584a8dac7189588e9445934d74743e80e6e186781b7be037a366a25c212\": container with ID starting with 6bd8d584a8dac7189588e9445934d74743e80e6e186781b7be037a366a25c212 not found: ID does not exist"
Sep 18 18:13:40.721935 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.721914    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"]
Sep 18 18:13:40.729194 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.729163    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-609aba/simulator-w0-609aba-kserve-68678954c4-knfzv"]
Sep 18 18:13:40.743619 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.743597    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"]
Sep 18 18:13:40.748188 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:40.748158    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-609aba/distinct-w0-609aba-kserve-68bf856846-v4tr9"]
Sep 18 18:13:42.046317 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:42.046286    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c3d631c-0aac-4a24-8e64-0ef855ccb8a0" path="/var/lib/kubelet/pods/4c3d631c-0aac-4a24-8e64-0ef855ccb8a0/volumes"
Sep 18 18:13:42.046699 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:13:42.046674    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55e9632e-d0c7-456d-a2f8-a3d998e8cd29" path="/var/lib/kubelet/pods/55e9632e-d0c7-456d-a2f8-a3d998e8cd29/volumes"
Sep 18 18:14:02.680370 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:02.680322    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 18:14:02.680813 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:02.680381    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm" podUID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 18:14:03.224760 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.224721    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"]
Sep 18 18:14:03.225228 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.225210    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c3d631c-0aac-4a24-8e64-0ef855ccb8a0" containerName="main"
Sep 18 18:14:03.225228 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.225226    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c3d631c-0aac-4a24-8e64-0ef855ccb8a0" containerName="main"
Sep 18 18:14:03.225396 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.225241    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1fec7ec-ea60-4ae4-8ac6-4748b259986b" containerName="maas-api"
Sep 18 18:14:03.225396 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.225247    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1fec7ec-ea60-4ae4-8ac6-4748b259986b" containerName="maas-api"
Sep 18 18:14:03.225396 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.225252    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55e9632e-d0c7-456d-a2f8-a3d998e8cd29" containerName="main"
Sep 18 18:14:03.225396 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.225257    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="55e9632e-d0c7-456d-a2f8-a3d998e8cd29" containerName="main"
Sep 18 18:14:03.225396 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.225342    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c3d631c-0aac-4a24-8e64-0ef855ccb8a0" containerName="main"
Sep 18 18:14:03.225396 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.225349    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1fec7ec-ea60-4ae4-8ac6-4748b259986b" containerName="maas-api"
Sep 18 18:14:03.225396 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.225359    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="55e9632e-d0c7-456d-a2f8-a3d998e8cd29" containerName="main"
Sep 18 18:14:03.228668 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.228643    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.231364 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.231332    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-341bd4-openshift-default-dockercfg-stkxl\""
Sep 18 18:14:03.242129 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.242098    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"]
Sep 18 18:14:03.319072 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.319021    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c2r9\" (UniqueName: \"kubernetes.io/projected/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-kube-api-access-4c2r9\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.319270 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.319084    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-token\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.319270 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.319119    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-workload-certs\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.319270 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.319148    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-workload-socket\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.319270 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.319220    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istiod-ca-cert\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.319270 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.319269    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-envoy\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.319489 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.319303    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-credential-socket\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.319489 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.319401    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-data\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.319489 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.319458    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-podinfo\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.420570 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.420536    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-podinfo\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.420768 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.420619    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c2r9\" (UniqueName: \"kubernetes.io/projected/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-kube-api-access-4c2r9\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.420768 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.420645    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-token\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.420768 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.420665    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-workload-certs\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.420768 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.420685    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-workload-socket\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.420768 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.420709    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istiod-ca-cert\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.420768 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.420758    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-envoy\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.421115 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.420799    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-credential-socket\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.421115 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.420829    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-data\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.421257 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.421108    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-workload-certs\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.421257 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.421172    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-data\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.421441 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.421409    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-credential-socket\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.421622 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.421596    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istiod-ca-cert\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.421705 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.421666    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-workload-socket\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.423238 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.423213    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-envoy\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.423408 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.423390    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-podinfo\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.429586 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.429561    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-token\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.429725 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.429706    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c2r9\" (UniqueName: \"kubernetes.io/projected/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-kube-api-access-4c2r9\") pod \"e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") " pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.542118 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.542025    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:03.842691 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.842668    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"]
Sep 18 18:14:03.843007 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.842710    2568 generic.go:356] "Generic (PLEG): container finished" podID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerID="2b818ac88151ba56bcb8a04f67de0a28d5887507426ae4602b36ee8e08edaa74" exitCode=0
Sep 18 18:14:03.843007 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.842724    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm" event={"ID":"4e083287-05ce-4370-b6d9-eb248cb8af8f","Type":"ContainerDied","Data":"2b818ac88151ba56bcb8a04f67de0a28d5887507426ae4602b36ee8e08edaa74"}
Sep 18 18:14:03.843326 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.843303    2568 scope.go:117] "RemoveContainer" containerID="2b818ac88151ba56bcb8a04f67de0a28d5887507426ae4602b36ee8e08edaa74"
Sep 18 18:14:03.844590 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.844550    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:14:03.844682 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.844626    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:14:03.844682 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.844663    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:14:03.845217 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:14:03.845123    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5c0e9bd_e389_4c85_9f50_75c3cae1c218.slice/crio-336ddb3af1b8738bc4ea29ed6a039e8086fdcaae9edeab4411b6abb42ffbe681 WatchSource:0}: Error finding container 336ddb3af1b8738bc4ea29ed6a039e8086fdcaae9edeab4411b6abb42ffbe681: Status 404 returned error can't find the container with id 336ddb3af1b8738bc4ea29ed6a039e8086fdcaae9edeab4411b6abb42ffbe681
Sep 18 18:14:03.847224 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.847194    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:14:03.847335 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.847259    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:14:03.847335 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:03.847289    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:14:04.850559 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:04.850521    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq" event={"ID":"b5c0e9bd-e389-4c85-9f50-75c3cae1c218","Type":"ContainerStarted","Data":"f002ba0910ba0f02e461fc12d01ccf1bff92581a4dcfa7bf6a4011641ec14488"}
Sep 18 18:14:04.851020 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:04.850583    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq" event={"ID":"b5c0e9bd-e389-4c85-9f50-75c3cae1c218","Type":"ContainerStarted","Data":"336ddb3af1b8738bc4ea29ed6a039e8086fdcaae9edeab4411b6abb42ffbe681"}
Sep 18 18:14:04.852690 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:04.852667    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm" event={"ID":"4e083287-05ce-4370-b6d9-eb248cb8af8f","Type":"ContainerStarted","Data":"1b33f07f4a0f76acb47f16eb83929cd61dad3c6c4db354cb563d02ea6fba328b"}
Sep 18 18:14:04.871309 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:04.871248    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq" podStartSLOduration=1.8712293070000001 podStartE2EDuration="1.871229307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:04.869207414 +0000 UTC m=+1448.947011315" watchObservedRunningTime="2026-09-18 18:14:04.871229307 +0000 UTC m=+1448.949033205"
Sep 18 18:14:05.094340 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:05.094307    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:14:05.094524 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:05.094463    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 18 18:14:05.094576 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:05.094516    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm" podUID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 18 18:14:05.423249 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:05.423212    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:14:05.542764 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:05.542722    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:05.548169 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:05.548139    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:05.872572 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:05.872480    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:05.877701 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:05.877672    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:06.095271 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:06.095233    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 18 18:14:06.095449 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:06.095292    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm" podUID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 18 18:14:07.095138 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:07.095100    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 18 18:14:07.095576 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:07.095156    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm" podUID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 18 18:14:08.095095 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:08.095052    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 18 18:14:08.095320 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:08.095109    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm" podUID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 18 18:14:09.094699 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:09.094662    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 18 18:14:09.094885 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:09.094722    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm" podUID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 18 18:14:10.094696 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.094653    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 18 18:14:10.095228 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.094715    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm" podUID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 18 18:14:10.716388 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.716348    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"]
Sep 18 18:14:10.720343 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.720272    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:10.722784 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.722759    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-341bd4-serving-cert\""
Sep 18 18:14:10.733255 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.733228    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"]
Sep 18 18:14:10.803729 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.803687    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmc7v\" (UniqueName: \"kubernetes.io/projected/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-kube-api-access-xmc7v\") pod \"maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:10.803729 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.803734    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:10.804016 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.803795    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:10.804016 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.803909    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:10.905912 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.905871    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:10.906264 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.906244    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmc7v\" (UniqueName: \"kubernetes.io/projected/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-kube-api-access-xmc7v\") pod \"maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:10.906434 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.906418    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:10.906558 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.906545    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:10.906742 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.906713    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:10.912265 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.910556    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:10.912265 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.912036    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:10.916273 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:10.916247    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmc7v\" (UniqueName: \"kubernetes.io/projected/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-kube-api-access-xmc7v\") pod \"maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:11.036421 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:11.036329    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:11.101057 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:11.101017    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:14:11.101971 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:11.101944    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:14:11.103031 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:11.103006    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:14:11.318718 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:11.318632    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"]
Sep 18 18:14:11.322292 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:14:11.322258    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99dda42d_cc46_4c2f_b4a4_08d979dfd7b5.slice/crio-b2f657aa88a0b9ba5f28fbcaac8f453edc7467a52b2120d10ed40602a0022f46 WatchSource:0}: Error finding container b2f657aa88a0b9ba5f28fbcaac8f453edc7467a52b2120d10ed40602a0022f46: Status 404 returned error can't find the container with id b2f657aa88a0b9ba5f28fbcaac8f453edc7467a52b2120d10ed40602a0022f46
Sep 18 18:14:11.324025 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:11.324006    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:14:11.917193 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:11.917152    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb" event={"ID":"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5","Type":"ContainerStarted","Data":"66ca0e49873d29d213b7f5983ecd4fa1c37076775ee038bfa1a5923a67e46ed8"}
Sep 18 18:14:11.917401 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:11.917377    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:11.917549 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:11.917416    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb" event={"ID":"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5","Type":"ContainerStarted","Data":"b2f657aa88a0b9ba5f28fbcaac8f453edc7467a52b2120d10ed40602a0022f46"}
Sep 18 18:14:11.934194 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:11.934123    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb" podStartSLOduration=1.5656743739999999 podStartE2EDuration="1.934106028s" totalImagesPullingTime="368.431654ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:11.93287743 +0000 UTC m=+1456.010681332" watchObservedRunningTime="2026-09-18 18:14:11.934106028 +0000 UTC m=+1456.011909927"
Sep 18 18:14:17.928544 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:17.928511    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:14:29.026127 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:29.026093    2568 generic.go:356] "Generic (PLEG): container finished" podID="b5c0e9bd-e389-4c85-9f50-75c3cae1c218" containerID="f002ba0910ba0f02e461fc12d01ccf1bff92581a4dcfa7bf6a4011641ec14488" exitCode=0
Sep 18 18:14:29.026663 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:29.026134    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq" event={"ID":"b5c0e9bd-e389-4c85-9f50-75c3cae1c218","Type":"ContainerDied","Data":"f002ba0910ba0f02e461fc12d01ccf1bff92581a4dcfa7bf6a4011641ec14488"}
Sep 18 18:14:29.026711 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:29.026687    2568 scope.go:117] "RemoveContainer" containerID="f002ba0910ba0f02e461fc12d01ccf1bff92581a4dcfa7bf6a4011641ec14488"
Sep 18 18:14:29.028077 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:29.028050    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:14:29.028165 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:29.028117    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:14:29.028165 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:29.028148    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:14:30.034164 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:30.034129    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq" event={"ID":"b5c0e9bd-e389-4c85-9f50-75c3cae1c218","Type":"ContainerStarted","Data":"46537d90c0b5c8224a0941b3ebde9b2212a5af9561cbb22193dc5ddec7cce77c"}
Sep 18 18:14:30.542575 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:30.542534    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:30.542767 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:30.542748    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 18 18:14:30.542814 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:30.542792    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq" podUID="b5c0e9bd-e389-4c85-9f50-75c3cae1c218" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 18 18:14:31.542669 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:31.542636    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 18 18:14:31.543065 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:31.542705    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq" podUID="b5c0e9bd-e389-4c85-9f50-75c3cae1c218" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 18 18:14:32.543079 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:32.543046    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 18 18:14:32.543497 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:32.543102    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq" podUID="b5c0e9bd-e389-4c85-9f50-75c3cae1c218" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 18 18:14:33.542389 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:33.542355    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 18 18:14:33.542568 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:33.542418    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq" podUID="b5c0e9bd-e389-4c85-9f50-75c3cae1c218" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 18 18:14:34.543416 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:34.543374    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 18 18:14:34.543825 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:34.543436    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq" podUID="b5c0e9bd-e389-4c85-9f50-75c3cae1c218" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 18 18:14:35.546703 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:35.546660    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:35.547187 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:35.546978    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:35.547659 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:35.547642    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:14:37.315307 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:37.315263    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:14:42.632455 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:42.632421    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"]
Sep 18 18:14:52.914857 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:52.914816    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:14:57.652801 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:57.652753    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd" podUID="7eaffe35-b703-4723-be05-fbbb3b80067c" containerName="main" containerID="cri-o://19c0c042d26aa8c052784226662f1cc4452b8dc4376f258ff27b67800e911207" gracePeriod=45
Sep 18 18:14:57.910089 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:57.910056    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:14:58.004104 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.004068    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-dshm\") pod \"7eaffe35-b703-4723-be05-fbbb3b80067c\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") "
Sep 18 18:14:58.004337 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.004122    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-model-cache\") pod \"7eaffe35-b703-4723-be05-fbbb3b80067c\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") "
Sep 18 18:14:58.004337 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.004211    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmq7c\" (UniqueName: \"kubernetes.io/projected/7eaffe35-b703-4723-be05-fbbb3b80067c-kube-api-access-gmq7c\") pod \"7eaffe35-b703-4723-be05-fbbb3b80067c\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") "
Sep 18 18:14:58.004337 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.004246    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-home\") pod \"7eaffe35-b703-4723-be05-fbbb3b80067c\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") "
Sep 18 18:14:58.004337 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.004270    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eaffe35-b703-4723-be05-fbbb3b80067c-tls-certs\") pod \"7eaffe35-b703-4723-be05-fbbb3b80067c\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") "
Sep 18 18:14:58.004569 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.004377    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-tmp-dir\") pod \"7eaffe35-b703-4723-be05-fbbb3b80067c\" (UID: \"7eaffe35-b703-4723-be05-fbbb3b80067c\") "
Sep 18 18:14:58.004569 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.004425    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-model-cache" (OuterVolumeSpecName: "model-cache") pod "7eaffe35-b703-4723-be05-fbbb3b80067c" (UID: "7eaffe35-b703-4723-be05-fbbb3b80067c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:14:58.004569 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.004531    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-home" (OuterVolumeSpecName: "home") pod "7eaffe35-b703-4723-be05-fbbb3b80067c" (UID: "7eaffe35-b703-4723-be05-fbbb3b80067c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:14:58.004738 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.004625    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7eaffe35-b703-4723-be05-fbbb3b80067c" (UID: "7eaffe35-b703-4723-be05-fbbb3b80067c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:14:58.004782 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.004773    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-home\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:14:58.004818 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.004790    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-tmp-dir\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:14:58.004818 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.004807    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-model-cache\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:14:58.006596 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.006567    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7eaffe35-b703-4723-be05-fbbb3b80067c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7eaffe35-b703-4723-be05-fbbb3b80067c" (UID: "7eaffe35-b703-4723-be05-fbbb3b80067c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:14:58.006683 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.006616    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-dshm" (OuterVolumeSpecName: "dshm") pod "7eaffe35-b703-4723-be05-fbbb3b80067c" (UID: "7eaffe35-b703-4723-be05-fbbb3b80067c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:14:58.006683 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.006640    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eaffe35-b703-4723-be05-fbbb3b80067c-kube-api-access-gmq7c" (OuterVolumeSpecName: "kube-api-access-gmq7c") pod "7eaffe35-b703-4723-be05-fbbb3b80067c" (UID: "7eaffe35-b703-4723-be05-fbbb3b80067c"). InnerVolumeSpecName "kube-api-access-gmq7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:14:58.090278 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.090247    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 18:14:58.097449 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.097424    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 18:14:58.105733 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.105708    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eaffe35-b703-4723-be05-fbbb3b80067c-dshm\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:14:58.105733 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.105731    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmq7c\" (UniqueName: \"kubernetes.io/projected/7eaffe35-b703-4723-be05-fbbb3b80067c-kube-api-access-gmq7c\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:14:58.105914 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.105742    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eaffe35-b703-4723-be05-fbbb3b80067c-tls-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:14:58.202870 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.202782    2568 generic.go:356] "Generic (PLEG): container finished" podID="7eaffe35-b703-4723-be05-fbbb3b80067c" containerID="19c0c042d26aa8c052784226662f1cc4452b8dc4376f258ff27b67800e911207" exitCode=0
Sep 18 18:14:58.202870 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.202827    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"
Sep 18 18:14:58.203062 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.202868    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd" event={"ID":"7eaffe35-b703-4723-be05-fbbb3b80067c","Type":"ContainerDied","Data":"19c0c042d26aa8c052784226662f1cc4452b8dc4376f258ff27b67800e911207"}
Sep 18 18:14:58.203062 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.202911    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd" event={"ID":"7eaffe35-b703-4723-be05-fbbb3b80067c","Type":"ContainerDied","Data":"885838990e8e2f02ace61d6cacedd1d92a16e8e0eedc34c77680d531fc14d5da"}
Sep 18 18:14:58.203062 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.202932    2568 scope.go:117] "RemoveContainer" containerID="19c0c042d26aa8c052784226662f1cc4452b8dc4376f258ff27b67800e911207"
Sep 18 18:14:58.212224 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.212203    2568 scope.go:117] "RemoveContainer" containerID="19c0c042d26aa8c052784226662f1cc4452b8dc4376f258ff27b67800e911207"
Sep 18 18:14:58.212541 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:14:58.212522    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19c0c042d26aa8c052784226662f1cc4452b8dc4376f258ff27b67800e911207\": container with ID starting with 19c0c042d26aa8c052784226662f1cc4452b8dc4376f258ff27b67800e911207 not found: ID does not exist" containerID="19c0c042d26aa8c052784226662f1cc4452b8dc4376f258ff27b67800e911207"
Sep 18 18:14:58.212594 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.212551    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19c0c042d26aa8c052784226662f1cc4452b8dc4376f258ff27b67800e911207"} err="failed to get container status \"19c0c042d26aa8c052784226662f1cc4452b8dc4376f258ff27b67800e911207\": rpc error: code = NotFound desc = could not find container \"19c0c042d26aa8c052784226662f1cc4452b8dc4376f258ff27b67800e911207\": container with ID starting with 19c0c042d26aa8c052784226662f1cc4452b8dc4376f258ff27b67800e911207 not found: ID does not exist"
Sep 18 18:14:58.218384 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.218357    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"]
Sep 18 18:14:58.224806 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:14:58.224780    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-03e512/e2e-conflict-b-e4d74f58-kserve-84486d5c94-t9knd"]
Sep 18 18:15:00.046019 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:00.045983    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eaffe35-b703-4723-be05-fbbb3b80067c" path="/var/lib/kubelet/pods/7eaffe35-b703-4723-be05-fbbb3b80067c/volumes"
Sep 18 18:15:00.148055 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:00.148014    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829255-l2xvh"]
Sep 18 18:15:00.148582 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:00.148561    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eaffe35-b703-4723-be05-fbbb3b80067c" containerName="main"
Sep 18 18:15:00.148688 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:00.148584    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eaffe35-b703-4723-be05-fbbb3b80067c" containerName="main"
Sep 18 18:15:00.148748 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:00.148700    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eaffe35-b703-4723-be05-fbbb3b80067c" containerName="main"
Sep 18 18:15:00.151472 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:00.151450    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-l2xvh"
Sep 18 18:15:00.154327 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:00.154300    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-88z7v\""
Sep 18 18:15:00.164952 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:00.164925    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-l2xvh"
Sep 18 18:15:00.166326 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:00.166295    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829255-l2xvh"]
Sep 18 18:15:00.169935 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:00.169905    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-53cde3-29829255-6g587"]
Sep 18 18:15:00.172462 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:00.172440    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-53cde3-29829255-6g587"
Sep 18 18:15:00.182169 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:00.182139    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-53cde3-29829255-6g587"]
Sep 18 18:15:00.206777 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:00.206752    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-53cde3-29829255-6g587"
Sep 18 18:15:00.451856 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:00.451828    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829255-l2xvh"]
Sep 18 18:15:00.454152 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:15:00.454118    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96e40a64_5ad0_4e18_b1e3_fac7ce4673a3.slice/crio-c8d9c3d8a8b51d2c8070ed730518baf905e1be07ff638f48a6e2cee16d9565b0 WatchSource:0}: Error finding container c8d9c3d8a8b51d2c8070ed730518baf905e1be07ff638f48a6e2cee16d9565b0: Status 404 returned error can't find the container with id c8d9c3d8a8b51d2c8070ed730518baf905e1be07ff638f48a6e2cee16d9565b0
Sep 18 18:15:00.494648 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:00.494623    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-53cde3-29829255-6g587"]
Sep 18 18:15:00.497025 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:15:00.496990    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9c995b0_2e3f_4661_9b8e_8a8efb2dd567.slice/crio-ac06deea3817723776319fe00d8282df651199448c38e550690ca1b6b83afc6c WatchSource:0}: Error finding container ac06deea3817723776319fe00d8282df651199448c38e550690ca1b6b83afc6c: Status 404 returned error can't find the container with id ac06deea3817723776319fe00d8282df651199448c38e550690ca1b6b83afc6c
Sep 18 18:15:01.221908 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:01.221871    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-l2xvh" event={"ID":"96e40a64-5ad0-4e18-b1e3-fac7ce4673a3","Type":"ContainerStarted","Data":"c8d9c3d8a8b51d2c8070ed730518baf905e1be07ff638f48a6e2cee16d9565b0"}
Sep 18 18:15:01.223054 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:01.223030    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-53cde3-29829255-6g587" event={"ID":"c9c995b0-2e3f-4661-9b8e-8a8efb2dd567","Type":"ContainerStarted","Data":"ac06deea3817723776319fe00d8282df651199448c38e550690ca1b6b83afc6c"}
Sep 18 18:15:01.250957 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:01.250917    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"]
Sep 18 18:15:04.243457 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:04.243423    2568 generic.go:356] "Generic (PLEG): container finished" podID="96e40a64-5ad0-4e18-b1e3-fac7ce4673a3" containerID="ee47d31408d49e11d94ef09b84b77512d7e97963006aab0f513b21d30d3cb251" exitCode=0
Sep 18 18:15:04.243886 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:04.243514    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-l2xvh" event={"ID":"96e40a64-5ad0-4e18-b1e3-fac7ce4673a3","Type":"ContainerDied","Data":"ee47d31408d49e11d94ef09b84b77512d7e97963006aab0f513b21d30d3cb251"}
Sep 18 18:15:04.245050 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:04.245025    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-53cde3-29829255-6g587" event={"ID":"c9c995b0-2e3f-4661-9b8e-8a8efb2dd567","Type":"ContainerStarted","Data":"f6ccd687aaaf04ae5eb4a83e720b5d9d540e911cb98eff2edf94a1a9c430d4dc"}
Sep 18 18:15:04.273443 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:04.273390    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-53cde3-29829255-6g587" podStartSLOduration=0.883722776 podStartE2EDuration="4.27337332s" totalImagesPullingTime="3.389650544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:15:04.272150213 +0000 UTC m=+1508.349954113" watchObservedRunningTime="2026-09-18 18:15:04.27337332 +0000 UTC m=+1508.351177218"
Sep 18 18:15:05.252466 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:05.252429    2568 generic.go:356] "Generic (PLEG): container finished" podID="c9c995b0-2e3f-4661-9b8e-8a8efb2dd567" containerID="f6ccd687aaaf04ae5eb4a83e720b5d9d540e911cb98eff2edf94a1a9c430d4dc" exitCode=0
Sep 18 18:15:05.252929 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:05.252526    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-53cde3-29829255-6g587" event={"ID":"c9c995b0-2e3f-4661-9b8e-8a8efb2dd567","Type":"ContainerDied","Data":"f6ccd687aaaf04ae5eb4a83e720b5d9d540e911cb98eff2edf94a1a9c430d4dc"}
Sep 18 18:15:05.395281 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:05.395251    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-l2xvh"
Sep 18 18:15:06.260048 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:06.260014    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-l2xvh"
Sep 18 18:15:06.260561 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:06.260046    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829255-l2xvh" event={"ID":"96e40a64-5ad0-4e18-b1e3-fac7ce4673a3","Type":"ContainerDied","Data":"c8d9c3d8a8b51d2c8070ed730518baf905e1be07ff638f48a6e2cee16d9565b0"}
Sep 18 18:15:06.260561 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:06.260108    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8d9c3d8a8b51d2c8070ed730518baf905e1be07ff638f48a6e2cee16d9565b0"
Sep 18 18:15:06.408615 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:06.408590    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-53cde3-29829255-6g587"
Sep 18 18:15:07.267990 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:07.267953    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-53cde3-29829255-6g587"
Sep 18 18:15:07.267990 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:07.267981    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-53cde3-29829255-6g587" event={"ID":"c9c995b0-2e3f-4661-9b8e-8a8efb2dd567","Type":"ContainerDied","Data":"ac06deea3817723776319fe00d8282df651199448c38e550690ca1b6b83afc6c"}
Sep 18 18:15:07.268464 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:07.268026    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac06deea3817723776319fe00d8282df651199448c38e550690ca1b6b83afc6c"
Sep 18 18:15:15.033917 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:15.033828    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:15:16.271251 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.271207    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl" podUID="ed102bd5-e4f3-4963-8a25-f9996157762e" containerName="main" containerID="cri-o://8f17dbdbb9441b8c2d8f3926dc14807aa9ea3905d864ad0a8a9969fca518debb" gracePeriod=45
Sep 18 18:15:16.529914 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.529838    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:15:16.607260 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.607226    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-model-cache\") pod \"ed102bd5-e4f3-4963-8a25-f9996157762e\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") "
Sep 18 18:15:16.607413 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.607270    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ggxb\" (UniqueName: \"kubernetes.io/projected/ed102bd5-e4f3-4963-8a25-f9996157762e-kube-api-access-7ggxb\") pod \"ed102bd5-e4f3-4963-8a25-f9996157762e\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") "
Sep 18 18:15:16.607413 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.607315    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-home\") pod \"ed102bd5-e4f3-4963-8a25-f9996157762e\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") "
Sep 18 18:15:16.607413 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.607355    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-dshm\") pod \"ed102bd5-e4f3-4963-8a25-f9996157762e\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") "
Sep 18 18:15:16.607413 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.607396    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-tmp-dir\") pod \"ed102bd5-e4f3-4963-8a25-f9996157762e\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") "
Sep 18 18:15:16.607642 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.607468    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed102bd5-e4f3-4963-8a25-f9996157762e-tls-certs\") pod \"ed102bd5-e4f3-4963-8a25-f9996157762e\" (UID: \"ed102bd5-e4f3-4963-8a25-f9996157762e\") "
Sep 18 18:15:16.607642 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.607579    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-model-cache" (OuterVolumeSpecName: "model-cache") pod "ed102bd5-e4f3-4963-8a25-f9996157762e" (UID: "ed102bd5-e4f3-4963-8a25-f9996157762e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:15:16.607763 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.607658    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-home" (OuterVolumeSpecName: "home") pod "ed102bd5-e4f3-4963-8a25-f9996157762e" (UID: "ed102bd5-e4f3-4963-8a25-f9996157762e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:15:16.607763 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.607700    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ed102bd5-e4f3-4963-8a25-f9996157762e" (UID: "ed102bd5-e4f3-4963-8a25-f9996157762e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:15:16.607907 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.607875    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-model-cache\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:15:16.607907 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.607897    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-home\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:15:16.608020 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.607911    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-tmp-dir\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:15:16.610191 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.610139    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed102bd5-e4f3-4963-8a25-f9996157762e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ed102bd5-e4f3-4963-8a25-f9996157762e" (UID: "ed102bd5-e4f3-4963-8a25-f9996157762e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:15:16.610191 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.610162    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed102bd5-e4f3-4963-8a25-f9996157762e-kube-api-access-7ggxb" (OuterVolumeSpecName: "kube-api-access-7ggxb") pod "ed102bd5-e4f3-4963-8a25-f9996157762e" (UID: "ed102bd5-e4f3-4963-8a25-f9996157762e"). InnerVolumeSpecName "kube-api-access-7ggxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:15:16.610355 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.610255    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-dshm" (OuterVolumeSpecName: "dshm") pod "ed102bd5-e4f3-4963-8a25-f9996157762e" (UID: "ed102bd5-e4f3-4963-8a25-f9996157762e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:15:16.708664 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.708622    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed102bd5-e4f3-4963-8a25-f9996157762e-tls-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:15:16.708664 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.708654    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ggxb\" (UniqueName: \"kubernetes.io/projected/ed102bd5-e4f3-4963-8a25-f9996157762e-kube-api-access-7ggxb\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:15:16.708664 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:16.708666    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed102bd5-e4f3-4963-8a25-f9996157762e-dshm\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:15:17.336925 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:17.336886    2568 generic.go:356] "Generic (PLEG): container finished" podID="ed102bd5-e4f3-4963-8a25-f9996157762e" containerID="8f17dbdbb9441b8c2d8f3926dc14807aa9ea3905d864ad0a8a9969fca518debb" exitCode=0
Sep 18 18:15:17.336925 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:17.336927    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"
Sep 18 18:15:17.337458 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:17.336953    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl" event={"ID":"ed102bd5-e4f3-4963-8a25-f9996157762e","Type":"ContainerDied","Data":"8f17dbdbb9441b8c2d8f3926dc14807aa9ea3905d864ad0a8a9969fca518debb"}
Sep 18 18:15:17.337458 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:17.337008    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl" event={"ID":"ed102bd5-e4f3-4963-8a25-f9996157762e","Type":"ContainerDied","Data":"47f768af58b6cfccaf79b011154ec798a3ba7ba14d3e3cb25f375e34e58b2ee2"}
Sep 18 18:15:17.337458 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:17.337031    2568 scope.go:117] "RemoveContainer" containerID="8f17dbdbb9441b8c2d8f3926dc14807aa9ea3905d864ad0a8a9969fca518debb"
Sep 18 18:15:17.349833 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:17.349807    2568 scope.go:117] "RemoveContainer" containerID="8f17dbdbb9441b8c2d8f3926dc14807aa9ea3905d864ad0a8a9969fca518debb"
Sep 18 18:15:17.350160 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:15:17.350141    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f17dbdbb9441b8c2d8f3926dc14807aa9ea3905d864ad0a8a9969fca518debb\": container with ID starting with 8f17dbdbb9441b8c2d8f3926dc14807aa9ea3905d864ad0a8a9969fca518debb not found: ID does not exist" containerID="8f17dbdbb9441b8c2d8f3926dc14807aa9ea3905d864ad0a8a9969fca518debb"
Sep 18 18:15:17.350272 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:17.350169    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f17dbdbb9441b8c2d8f3926dc14807aa9ea3905d864ad0a8a9969fca518debb"} err="failed to get container status \"8f17dbdbb9441b8c2d8f3926dc14807aa9ea3905d864ad0a8a9969fca518debb\": rpc error: code = NotFound desc = could not find container \"8f17dbdbb9441b8c2d8f3926dc14807aa9ea3905d864ad0a8a9969fca518debb\": container with ID starting with 8f17dbdbb9441b8c2d8f3926dc14807aa9ea3905d864ad0a8a9969fca518debb not found: ID does not exist"
Sep 18 18:15:17.372326 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:17.372291    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"]
Sep 18 18:15:17.388443 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:17.388401    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-f7e72d9f/e2e-lifecycle-model-f7e72d9f-kserve-7bc68c68dc-s2rjl"]
Sep 18 18:15:18.045552 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:18.045519    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed102bd5-e4f3-4963-8a25-f9996157762e" path="/var/lib/kubelet/pods/ed102bd5-e4f3-4963-8a25-f9996157762e/volumes"
Sep 18 18:15:52.614464 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:15:52.614430    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:16:03.257214 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.257142    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"]
Sep 18 18:16:03.257804 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.257783    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9c995b0-2e3f-4661-9b8e-8a8efb2dd567" containerName="cleanup"
Sep 18 18:16:03.257898 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.257808    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9c995b0-2e3f-4661-9b8e-8a8efb2dd567" containerName="cleanup"
Sep 18 18:16:03.257898 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.257838    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed102bd5-e4f3-4963-8a25-f9996157762e" containerName="main"
Sep 18 18:16:03.257898 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.257846    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed102bd5-e4f3-4963-8a25-f9996157762e" containerName="main"
Sep 18 18:16:03.257898 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.257859    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96e40a64-5ad0-4e18-b1e3-fac7ce4673a3" containerName="cleanup"
Sep 18 18:16:03.257898 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.257867    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="96e40a64-5ad0-4e18-b1e3-fac7ce4673a3" containerName="cleanup"
Sep 18 18:16:03.258138 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.257990    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed102bd5-e4f3-4963-8a25-f9996157762e" containerName="main"
Sep 18 18:16:03.258138 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.258003    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9c995b0-2e3f-4661-9b8e-8a8efb2dd567" containerName="cleanup"
Sep 18 18:16:03.258138 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.258016    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="96e40a64-5ad0-4e18-b1e3-fac7ce4673a3" containerName="cleanup"
Sep 18 18:16:03.262612 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.262589    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.269425 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.269312    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-03e512\"/\"distinct-2-w4-03e512-kserve-self-signed-certs\""
Sep 18 18:16:03.277638 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.277608    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"]
Sep 18 18:16:03.388668 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.388635    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x85j\" (UniqueName: \"kubernetes.io/projected/432c5685-7f13-45f7-b5a3-44947c7d9c52-kube-api-access-2x85j\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.388668 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.388677    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-model-cache\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.388878 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.388782    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/432c5685-7f13-45f7-b5a3-44947c7d9c52-tls-certs\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.388878 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.388847    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-home\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.388954 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.388876    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-dshm\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.388954 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.388905    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-tmp-dir\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.489759 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.489714    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/432c5685-7f13-45f7-b5a3-44947c7d9c52-tls-certs\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.489954 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.489928    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-home\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.490362 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.490328    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-dshm\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.490513 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.490371    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-home\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.490513 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.490418    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-tmp-dir\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.490513 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.490470    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x85j\" (UniqueName: \"kubernetes.io/projected/432c5685-7f13-45f7-b5a3-44947c7d9c52-kube-api-access-2x85j\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.490694 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.490499    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-model-cache\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.490794 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.490770    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-tmp-dir\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.490909 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.490887    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-model-cache\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.493001 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.492978    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-dshm\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.493150 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.493132    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/432c5685-7f13-45f7-b5a3-44947c7d9c52-tls-certs\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.502976 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.502946    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x85j\" (UniqueName: \"kubernetes.io/projected/432c5685-7f13-45f7-b5a3-44947c7d9c52-kube-api-access-2x85j\") pod \"distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") " pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.572464 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.572373    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:03.850222 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:03.850169    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"]
Sep 18 18:16:03.851219 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:16:03.851173    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod432c5685_7f13_45f7_b5a3_44947c7d9c52.slice/crio-89763bda4c0c6208dad50d86f6585945d2b141fccdbbac229e8f92d19cb1925d WatchSource:0}: Error finding container 89763bda4c0c6208dad50d86f6585945d2b141fccdbbac229e8f92d19cb1925d: Status 404 returned error can't find the container with id 89763bda4c0c6208dad50d86f6585945d2b141fccdbbac229e8f92d19cb1925d
Sep 18 18:16:04.605413 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:04.605372    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z" event={"ID":"432c5685-7f13-45f7-b5a3-44947c7d9c52","Type":"ContainerStarted","Data":"6f5563f6d259221247ae7af3be79be76ea8bedccea202e4e3d438caf363f723a"}
Sep 18 18:16:04.605413 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:04.605421    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z" event={"ID":"432c5685-7f13-45f7-b5a3-44947c7d9c52","Type":"ContainerStarted","Data":"89763bda4c0c6208dad50d86f6585945d2b141fccdbbac229e8f92d19cb1925d"}
Sep 18 18:16:04.633328 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:04.633270    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z" podStartSLOduration=1.633252949 podStartE2EDuration="1.633252949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:16:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:16:04.631749216 +0000 UTC m=+1568.709553114" watchObservedRunningTime="2026-09-18 18:16:04.633252949 +0000 UTC m=+1568.711056848"
Sep 18 18:16:10.626685 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:10.626652    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:16:13.572740 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:13.572704    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:13.573215 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:13.572838    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:13.586158 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:13.586132    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:13.671240 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:13.671208    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:16:36.814055 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:16:36.814020    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:17:00.819999 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:00.819920    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:17:12.205130 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.205097    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"]
Sep 18 18:17:12.209193 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.209144    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:12.211821 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.211799    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-e9a4f9-serving-cert\""
Sep 18 18:17:12.218109 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.218084    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"]
Sep 18 18:17:12.306526 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.306486    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fkt6\" (UniqueName: \"kubernetes.io/projected/028d2288-c88d-45b7-a9b4-8276120ccc19-kube-api-access-8fkt6\") pod \"maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:12.306709 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.306594    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/028d2288-c88d-45b7-a9b4-8276120ccc19-maas-api-tls\") pod \"maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:12.306768 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.306705    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/028d2288-c88d-45b7-a9b4-8276120ccc19-openshift-service-ca\") pod \"maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:12.306768 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.306759    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/028d2288-c88d-45b7-a9b4-8276120ccc19-metrics-certs\") pod \"maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:12.408453 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.408410    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fkt6\" (UniqueName: \"kubernetes.io/projected/028d2288-c88d-45b7-a9b4-8276120ccc19-kube-api-access-8fkt6\") pod \"maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:12.408657 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.408479    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/028d2288-c88d-45b7-a9b4-8276120ccc19-maas-api-tls\") pod \"maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:12.408657 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.408518    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/028d2288-c88d-45b7-a9b4-8276120ccc19-openshift-service-ca\") pod \"maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:12.408657 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.408545    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/028d2288-c88d-45b7-a9b4-8276120ccc19-metrics-certs\") pod \"maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:12.408657 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:17:12.408642    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-serving-cert: secret "maas-api-e2e-api-a-e9a4f9-serving-cert" not found
Sep 18 18:17:12.408854 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:17:12.408738    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/028d2288-c88d-45b7-a9b4-8276120ccc19-maas-api-tls podName:028d2288-c88d-45b7-a9b4-8276120ccc19 nodeName:}" failed. No retries permitted until 2026-09-18 18:17:12.908714947 +0000 UTC m=+1636.986518832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/028d2288-c88d-45b7-a9b4-8276120ccc19-maas-api-tls") pod "maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm" (UID: "028d2288-c88d-45b7-a9b4-8276120ccc19") : secret "maas-api-e2e-api-a-e9a4f9-serving-cert" not found
Sep 18 18:17:12.409296 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.409278    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/028d2288-c88d-45b7-a9b4-8276120ccc19-openshift-service-ca\") pod \"maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:12.411214 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.411192    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/028d2288-c88d-45b7-a9b4-8276120ccc19-metrics-certs\") pod \"maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:12.418514 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.418490    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fkt6\" (UniqueName: \"kubernetes.io/projected/028d2288-c88d-45b7-a9b4-8276120ccc19-kube-api-access-8fkt6\") pod \"maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:12.914747 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.914717    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/028d2288-c88d-45b7-a9b4-8276120ccc19-maas-api-tls\") pod \"maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:12.917235 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:12.917216    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/028d2288-c88d-45b7-a9b4-8276120ccc19-maas-api-tls\") pod \"maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:13.120584 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:13.120552    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:13.394094 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:13.394040    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"]
Sep 18 18:17:13.395400 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:17:13.395368    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod028d2288_c88d_45b7_a9b4_8276120ccc19.slice/crio-db0cf343cd942ec093da8bffba4e4ca9d8bafcca8e72fe080386d61ec0438b64 WatchSource:0}: Error finding container db0cf343cd942ec093da8bffba4e4ca9d8bafcca8e72fe080386d61ec0438b64: Status 404 returned error can't find the container with id db0cf343cd942ec093da8bffba4e4ca9d8bafcca8e72fe080386d61ec0438b64
Sep 18 18:17:14.003579 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:14.003545    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm" event={"ID":"028d2288-c88d-45b7-a9b4-8276120ccc19","Type":"ContainerStarted","Data":"828db8f0dcbccb37895662eb28ab528bb52465d10a38c153a030d10261004395"}
Sep 18 18:17:14.003760 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:14.003597    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm" event={"ID":"028d2288-c88d-45b7-a9b4-8276120ccc19","Type":"ContainerStarted","Data":"db0cf343cd942ec093da8bffba4e4ca9d8bafcca8e72fe080386d61ec0438b64"}
Sep 18 18:17:14.003760 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:14.003635    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:14.020523 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:14.020476    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm" podStartSLOduration=1.543671128 podStartE2EDuration="2.020460881s" totalImagesPullingTime="476.789753ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:17:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:17:14.018002195 +0000 UTC m=+1638.095806094" watchObservedRunningTime="2026-09-18 18:17:14.020460881 +0000 UTC m=+1638.098264780"
Sep 18 18:17:20.016008 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:20.015977    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:17:26.817863 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:26.817831    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:17:39.230463 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.230432    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"]
Sep 18 18:17:39.234471 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.234448    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.236613 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.236590    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-005032-openshift-default-dockercfg-sscq2\""
Sep 18 18:17:39.246628 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.246600    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"]
Sep 18 18:17:39.288693 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.288651    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-credential-socket\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.288840 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.288704    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-data\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.288840 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.288739    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-token\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.288840 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.288794    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xczxb\" (UniqueName: \"kubernetes.io/projected/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-kube-api-access-xczxb\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.288840 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.288819    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-podinfo\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.288990 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.288863    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istiod-ca-cert\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.288990 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.288914    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-workload-socket\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.289058 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.289013    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-envoy\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.289137 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.289062    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-workload-certs\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.390608 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.390574    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-workload-socket\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.390771 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.390633    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-envoy\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.390771 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.390673    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-workload-certs\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.390771 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.390698    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-credential-socket\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.390771 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.390730    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-data\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.390771 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.390755    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-token\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.391029 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.390792    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xczxb\" (UniqueName: \"kubernetes.io/projected/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-kube-api-access-xczxb\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.391029 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.390817    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-podinfo\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.391029 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.390858    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istiod-ca-cert\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.391029 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.390955    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-workload-socket\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.391029 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.391023    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-credential-socket\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.391392 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.391062    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-workload-certs\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.391392 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.391373    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-data\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.391504 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.391488    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istiod-ca-cert\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.393068 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.393046    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-envoy\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.393318 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.393300    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-podinfo\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.398585 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.398561    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xczxb\" (UniqueName: \"kubernetes.io/projected/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-kube-api-access-xczxb\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.398857 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.398840    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-token\") pod \"e2e-api-b-005032-openshift-default-6666b966b-nfw84\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") " pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.546648 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.546566    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:39.824942 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.824913    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"]
Sep 18 18:17:39.826644 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:17:39.826620    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod861b37c6_902d_48f6_8ffa_9be5cf4eb5cc.slice/crio-18c5638c326c828b1038e1f92d120a30399ed2f223593744bdac36644fd3fc3c WatchSource:0}: Error finding container 18c5638c326c828b1038e1f92d120a30399ed2f223593744bdac36644fd3fc3c: Status 404 returned error can't find the container with id 18c5638c326c828b1038e1f92d120a30399ed2f223593744bdac36644fd3fc3c
Sep 18 18:17:39.828941 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.828909    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:17:39.829021 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.828976    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:17:39.829021 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:39.829002    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:17:40.154781 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:40.154680    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84" event={"ID":"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc","Type":"ContainerStarted","Data":"99916783bcdd3cbb3eb58805d2796c56ace7da60ca9ecbd996fe5fbbe318db8c"}
Sep 18 18:17:40.154781 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:40.154751    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84" event={"ID":"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc","Type":"ContainerStarted","Data":"18c5638c326c828b1038e1f92d120a30399ed2f223593744bdac36644fd3fc3c"}
Sep 18 18:17:40.175512 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:40.175447    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84" podStartSLOduration=1.1754263520000001 podStartE2EDuration="1.175426352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:17:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:17:40.173778826 +0000 UTC m=+1664.251582746" watchObservedRunningTime="2026-09-18 18:17:40.175426352 +0000 UTC m=+1664.253230252"
Sep 18 18:17:40.547733 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:40.547693    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:40.552989 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:40.552961    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:41.162152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:41.162118    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:41.163042 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:41.163017    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:17:46.872498 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:46.872463    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"]
Sep 18 18:17:46.876594 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:46.876571    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:46.879049 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:46.879028    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-005032-serving-cert\""
Sep 18 18:17:46.888305 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:46.888274    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"]
Sep 18 18:17:46.974287 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:46.974246    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58714b51-9041-4005-8496-7c628f8f6c01-openshift-service-ca\") pod \"maas-api-e2e-api-b-005032-66884b6f5b-94cdq\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:46.974495 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:46.974301    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r97m4\" (UniqueName: \"kubernetes.io/projected/58714b51-9041-4005-8496-7c628f8f6c01-kube-api-access-r97m4\") pod \"maas-api-e2e-api-b-005032-66884b6f5b-94cdq\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:46.974571 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:46.974498    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58714b51-9041-4005-8496-7c628f8f6c01-metrics-certs\") pod \"maas-api-e2e-api-b-005032-66884b6f5b-94cdq\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:46.974571 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:46.974542    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58714b51-9041-4005-8496-7c628f8f6c01-maas-api-tls\") pod \"maas-api-e2e-api-b-005032-66884b6f5b-94cdq\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:47.075708 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:47.075675    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58714b51-9041-4005-8496-7c628f8f6c01-openshift-service-ca\") pod \"maas-api-e2e-api-b-005032-66884b6f5b-94cdq\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:47.075891 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:47.075720    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r97m4\" (UniqueName: \"kubernetes.io/projected/58714b51-9041-4005-8496-7c628f8f6c01-kube-api-access-r97m4\") pod \"maas-api-e2e-api-b-005032-66884b6f5b-94cdq\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:47.075891 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:47.075840    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58714b51-9041-4005-8496-7c628f8f6c01-metrics-certs\") pod \"maas-api-e2e-api-b-005032-66884b6f5b-94cdq\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:47.075891 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:47.075877    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58714b51-9041-4005-8496-7c628f8f6c01-maas-api-tls\") pod \"maas-api-e2e-api-b-005032-66884b6f5b-94cdq\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:47.076071 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:17:47.076013    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-005032-serving-cert: secret "maas-api-e2e-api-b-005032-serving-cert" not found
Sep 18 18:17:47.076129 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:17:47.076094    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58714b51-9041-4005-8496-7c628f8f6c01-maas-api-tls podName:58714b51-9041-4005-8496-7c628f8f6c01 nodeName:}" failed. No retries permitted until 2026-09-18 18:17:47.576073859 +0000 UTC m=+1671.653877737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/58714b51-9041-4005-8496-7c628f8f6c01-maas-api-tls") pod "maas-api-e2e-api-b-005032-66884b6f5b-94cdq" (UID: "58714b51-9041-4005-8496-7c628f8f6c01") : secret "maas-api-e2e-api-b-005032-serving-cert" not found
Sep 18 18:17:47.076495 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:47.076471    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58714b51-9041-4005-8496-7c628f8f6c01-openshift-service-ca\") pod \"maas-api-e2e-api-b-005032-66884b6f5b-94cdq\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:47.078414 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:47.078392    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58714b51-9041-4005-8496-7c628f8f6c01-metrics-certs\") pod \"maas-api-e2e-api-b-005032-66884b6f5b-94cdq\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:47.085091 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:47.085062    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r97m4\" (UniqueName: \"kubernetes.io/projected/58714b51-9041-4005-8496-7c628f8f6c01-kube-api-access-r97m4\") pod \"maas-api-e2e-api-b-005032-66884b6f5b-94cdq\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:47.581263 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:47.581229    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58714b51-9041-4005-8496-7c628f8f6c01-maas-api-tls\") pod \"maas-api-e2e-api-b-005032-66884b6f5b-94cdq\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:47.583677 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:47.583649    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58714b51-9041-4005-8496-7c628f8f6c01-maas-api-tls\") pod \"maas-api-e2e-api-b-005032-66884b6f5b-94cdq\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:47.788858 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:47.788824    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:48.066799 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:48.066768    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"]
Sep 18 18:17:48.067114 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:17:48.066804    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58714b51_9041_4005_8496_7c628f8f6c01.slice/crio-43ef16636b8d5e0a80bd00056a777f5e9cc0a1ef9a13c4895fb55e760a129938 WatchSource:0}: Error finding container 43ef16636b8d5e0a80bd00056a777f5e9cc0a1ef9a13c4895fb55e760a129938: Status 404 returned error can't find the container with id 43ef16636b8d5e0a80bd00056a777f5e9cc0a1ef9a13c4895fb55e760a129938
Sep 18 18:17:48.203323 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:48.203288    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq" event={"ID":"58714b51-9041-4005-8496-7c628f8f6c01","Type":"ContainerStarted","Data":"43ef16636b8d5e0a80bd00056a777f5e9cc0a1ef9a13c4895fb55e760a129938"}
Sep 18 18:17:49.210661 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:49.210618    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq" event={"ID":"58714b51-9041-4005-8496-7c628f8f6c01","Type":"ContainerStarted","Data":"655be3c00fbbba458f8738dafca93f8b0016812201324c193a5a19ffddbe96d4"}
Sep 18 18:17:49.211053 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:49.210748    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:17:49.229518 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:49.229472    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq" podStartSLOduration=2.758261784 podStartE2EDuration="3.229456803s" totalImagesPullingTime="471.195019ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:17:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:17:49.226795939 +0000 UTC m=+1673.304599837" watchObservedRunningTime="2026-09-18 18:17:49.229456803 +0000 UTC m=+1673.307260769"
Sep 18 18:17:54.721283 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:54.721241    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:17:55.222425 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:17:55.222392    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:18:06.327424 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:06.327392    2568 generic.go:356] "Generic (PLEG): container finished" podID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerID="99916783bcdd3cbb3eb58805d2796c56ace7da60ca9ecbd996fe5fbbe318db8c" exitCode=0
Sep 18 18:18:06.327874 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:06.327467    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84" event={"ID":"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc","Type":"ContainerDied","Data":"99916783bcdd3cbb3eb58805d2796c56ace7da60ca9ecbd996fe5fbbe318db8c"}
Sep 18 18:18:06.327919 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:06.327906    2568 scope.go:117] "RemoveContainer" containerID="99916783bcdd3cbb3eb58805d2796c56ace7da60ca9ecbd996fe5fbbe318db8c"
Sep 18 18:18:06.329163 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:06.329133    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:18:06.329258 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:06.329223    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:18:06.329298 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:06.329256    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:18:07.336385 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:07.336351    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84" event={"ID":"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc","Type":"ContainerStarted","Data":"37abb756d3eb453dffa69df601f345fe9e053bbf2659b6b16b7056503a206fd8"}
Sep 18 18:18:07.547244 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:07.547214    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:18:07.548610 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:07.548579    2568 patch_prober.go:28] interesting pod/e2e-api-b-005032-openshift-default-6666b966b-nfw84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 18 18:18:07.548757 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:07.548643    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84" podUID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 18 18:18:08.547852 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:08.547823    2568 patch_prober.go:28] interesting pod/e2e-api-b-005032-openshift-default-6666b966b-nfw84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 18 18:18:08.548461 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:08.547928    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84" podUID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 18 18:18:09.547611 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:09.547565    2568 patch_prober.go:28] interesting pod/e2e-api-b-005032-openshift-default-6666b966b-nfw84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 18 18:18:09.547816 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:09.547632    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84" podUID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 18 18:18:10.547563 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:10.547484    2568 patch_prober.go:28] interesting pod/e2e-api-b-005032-openshift-default-6666b966b-nfw84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 18 18:18:10.547563 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:10.547539    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84" podUID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 18 18:18:11.162427 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:11.162382    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:18:11.547572 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:11.547485    2568 patch_prober.go:28] interesting pod/e2e-api-b-005032-openshift-default-6666b966b-nfw84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 18 18:18:11.547572 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:11.547541    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84" podUID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 18 18:18:12.546868 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:12.546830    2568 patch_prober.go:28] interesting pod/e2e-api-b-005032-openshift-default-6666b966b-nfw84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 18 18:18:12.547057 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:12.546895    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84" podUID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 18 18:18:13.551146 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:13.551118    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:18:13.552280 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:13.552257    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:18:28.220576 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:28.220543    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:18:30.439309 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:30.439278    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"]
Sep 18 18:18:30.439715 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:30.439490    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm" podUID="028d2288-c88d-45b7-a9b4-8276120ccc19" containerName="maas-api" containerID="cri-o://828db8f0dcbccb37895662eb28ab528bb52465d10a38c153a030d10261004395" gracePeriod=30
Sep 18 18:18:30.716822 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:30.716730    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:18:30.841263 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:30.841234    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/028d2288-c88d-45b7-a9b4-8276120ccc19-metrics-certs\") pod \"028d2288-c88d-45b7-a9b4-8276120ccc19\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") "
Sep 18 18:18:30.841437 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:30.841310    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fkt6\" (UniqueName: \"kubernetes.io/projected/028d2288-c88d-45b7-a9b4-8276120ccc19-kube-api-access-8fkt6\") pod \"028d2288-c88d-45b7-a9b4-8276120ccc19\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") "
Sep 18 18:18:30.841437 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:30.841421    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/028d2288-c88d-45b7-a9b4-8276120ccc19-maas-api-tls\") pod \"028d2288-c88d-45b7-a9b4-8276120ccc19\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") "
Sep 18 18:18:30.841550 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:30.841462    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/028d2288-c88d-45b7-a9b4-8276120ccc19-openshift-service-ca\") pod \"028d2288-c88d-45b7-a9b4-8276120ccc19\" (UID: \"028d2288-c88d-45b7-a9b4-8276120ccc19\") "
Sep 18 18:18:30.841880 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:30.841849    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/028d2288-c88d-45b7-a9b4-8276120ccc19-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "028d2288-c88d-45b7-a9b4-8276120ccc19" (UID: "028d2288-c88d-45b7-a9b4-8276120ccc19"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:18:30.843567 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:30.843538    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/028d2288-c88d-45b7-a9b4-8276120ccc19-kube-api-access-8fkt6" (OuterVolumeSpecName: "kube-api-access-8fkt6") pod "028d2288-c88d-45b7-a9b4-8276120ccc19" (UID: "028d2288-c88d-45b7-a9b4-8276120ccc19"). InnerVolumeSpecName "kube-api-access-8fkt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:18:30.843684 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:30.843605    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028d2288-c88d-45b7-a9b4-8276120ccc19-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "028d2288-c88d-45b7-a9b4-8276120ccc19" (UID: "028d2288-c88d-45b7-a9b4-8276120ccc19"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:18:30.843834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:30.843811    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028d2288-c88d-45b7-a9b4-8276120ccc19-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "028d2288-c88d-45b7-a9b4-8276120ccc19" (UID: "028d2288-c88d-45b7-a9b4-8276120ccc19"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:18:30.943013 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:30.942978    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/028d2288-c88d-45b7-a9b4-8276120ccc19-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:30.943013 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:30.943008    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/028d2288-c88d-45b7-a9b4-8276120ccc19-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:30.943013 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:30.943019    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/028d2288-c88d-45b7-a9b4-8276120ccc19-metrics-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:30.943291 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:30.943027    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fkt6\" (UniqueName: \"kubernetes.io/projected/028d2288-c88d-45b7-a9b4-8276120ccc19-kube-api-access-8fkt6\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:31.482118 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:31.482086    2568 generic.go:356] "Generic (PLEG): container finished" podID="028d2288-c88d-45b7-a9b4-8276120ccc19" containerID="828db8f0dcbccb37895662eb28ab528bb52465d10a38c153a030d10261004395" exitCode=0
Sep 18 18:18:31.482563 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:31.482130    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"
Sep 18 18:18:31.482563 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:31.482194    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm" event={"ID":"028d2288-c88d-45b7-a9b4-8276120ccc19","Type":"ContainerDied","Data":"828db8f0dcbccb37895662eb28ab528bb52465d10a38c153a030d10261004395"}
Sep 18 18:18:31.482563 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:31.482243    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm" event={"ID":"028d2288-c88d-45b7-a9b4-8276120ccc19","Type":"ContainerDied","Data":"db0cf343cd942ec093da8bffba4e4ca9d8bafcca8e72fe080386d61ec0438b64"}
Sep 18 18:18:31.482563 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:31.482264    2568 scope.go:117] "RemoveContainer" containerID="828db8f0dcbccb37895662eb28ab528bb52465d10a38c153a030d10261004395"
Sep 18 18:18:31.492387 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:31.492370    2568 scope.go:117] "RemoveContainer" containerID="828db8f0dcbccb37895662eb28ab528bb52465d10a38c153a030d10261004395"
Sep 18 18:18:31.492658 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:18:31.492639    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"828db8f0dcbccb37895662eb28ab528bb52465d10a38c153a030d10261004395\": container with ID starting with 828db8f0dcbccb37895662eb28ab528bb52465d10a38c153a030d10261004395 not found: ID does not exist" containerID="828db8f0dcbccb37895662eb28ab528bb52465d10a38c153a030d10261004395"
Sep 18 18:18:31.492724 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:31.492664    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"828db8f0dcbccb37895662eb28ab528bb52465d10a38c153a030d10261004395"} err="failed to get container status \"828db8f0dcbccb37895662eb28ab528bb52465d10a38c153a030d10261004395\": rpc error: code = NotFound desc = could not find container \"828db8f0dcbccb37895662eb28ab528bb52465d10a38c153a030d10261004395\": container with ID starting with 828db8f0dcbccb37895662eb28ab528bb52465d10a38c153a030d10261004395 not found: ID does not exist"
Sep 18 18:18:31.509794 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:31.509764    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"]
Sep 18 18:18:31.516615 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:31.516596    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e9a4f9-65786768dc-lt2jm"]
Sep 18 18:18:32.045313 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:32.045276    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="028d2288-c88d-45b7-a9b4-8276120ccc19" path="/var/lib/kubelet/pods/028d2288-c88d-45b7-a9b4-8276120ccc19/volumes"
Sep 18 18:18:38.374093 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:38.374060    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"]
Sep 18 18:18:38.391819 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:38.391787    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"]
Sep 18 18:18:50.926921 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:50.926881    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"]
Sep 18 18:18:50.927558 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:50.927139    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq" podUID="58714b51-9041-4005-8496-7c628f8f6c01" containerName="maas-api" containerID="cri-o://655be3c00fbbba458f8738dafca93f8b0016812201324c193a5a19ffddbe96d4" gracePeriod=30
Sep 18 18:18:51.193793 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.193699    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:18:51.259365 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.259333    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r97m4\" (UniqueName: \"kubernetes.io/projected/58714b51-9041-4005-8496-7c628f8f6c01-kube-api-access-r97m4\") pod \"58714b51-9041-4005-8496-7c628f8f6c01\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") "
Sep 18 18:18:51.259541 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.259376    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58714b51-9041-4005-8496-7c628f8f6c01-maas-api-tls\") pod \"58714b51-9041-4005-8496-7c628f8f6c01\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") "
Sep 18 18:18:51.259541 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.259395    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58714b51-9041-4005-8496-7c628f8f6c01-metrics-certs\") pod \"58714b51-9041-4005-8496-7c628f8f6c01\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") "
Sep 18 18:18:51.259541 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.259532    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58714b51-9041-4005-8496-7c628f8f6c01-openshift-service-ca\") pod \"58714b51-9041-4005-8496-7c628f8f6c01\" (UID: \"58714b51-9041-4005-8496-7c628f8f6c01\") "
Sep 18 18:18:51.260150 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.260116    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58714b51-9041-4005-8496-7c628f8f6c01-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "58714b51-9041-4005-8496-7c628f8f6c01" (UID: "58714b51-9041-4005-8496-7c628f8f6c01"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:18:51.261750 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.261717    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58714b51-9041-4005-8496-7c628f8f6c01-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "58714b51-9041-4005-8496-7c628f8f6c01" (UID: "58714b51-9041-4005-8496-7c628f8f6c01"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:18:51.261750 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.261728    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58714b51-9041-4005-8496-7c628f8f6c01-kube-api-access-r97m4" (OuterVolumeSpecName: "kube-api-access-r97m4") pod "58714b51-9041-4005-8496-7c628f8f6c01" (UID: "58714b51-9041-4005-8496-7c628f8f6c01"). InnerVolumeSpecName "kube-api-access-r97m4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:18:51.261932 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.261915    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58714b51-9041-4005-8496-7c628f8f6c01-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "58714b51-9041-4005-8496-7c628f8f6c01" (UID: "58714b51-9041-4005-8496-7c628f8f6c01"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:18:51.361384 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.361346    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58714b51-9041-4005-8496-7c628f8f6c01-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:51.361384 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.361374    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r97m4\" (UniqueName: \"kubernetes.io/projected/58714b51-9041-4005-8496-7c628f8f6c01-kube-api-access-r97m4\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:51.361384 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.361384    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58714b51-9041-4005-8496-7c628f8f6c01-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:51.361384 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.361392    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58714b51-9041-4005-8496-7c628f8f6c01-metrics-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:51.610565 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.610470    2568 generic.go:356] "Generic (PLEG): container finished" podID="58714b51-9041-4005-8496-7c628f8f6c01" containerID="655be3c00fbbba458f8738dafca93f8b0016812201324c193a5a19ffddbe96d4" exitCode=0
Sep 18 18:18:51.610565 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.610517    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"
Sep 18 18:18:51.610810 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.610559    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq" event={"ID":"58714b51-9041-4005-8496-7c628f8f6c01","Type":"ContainerDied","Data":"655be3c00fbbba458f8738dafca93f8b0016812201324c193a5a19ffddbe96d4"}
Sep 18 18:18:51.610810 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.610620    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq" event={"ID":"58714b51-9041-4005-8496-7c628f8f6c01","Type":"ContainerDied","Data":"43ef16636b8d5e0a80bd00056a777f5e9cc0a1ef9a13c4895fb55e760a129938"}
Sep 18 18:18:51.610810 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.610649    2568 scope.go:117] "RemoveContainer" containerID="655be3c00fbbba458f8738dafca93f8b0016812201324c193a5a19ffddbe96d4"
Sep 18 18:18:51.620666 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.620647    2568 scope.go:117] "RemoveContainer" containerID="655be3c00fbbba458f8738dafca93f8b0016812201324c193a5a19ffddbe96d4"
Sep 18 18:18:51.620948 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:18:51.620929    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"655be3c00fbbba458f8738dafca93f8b0016812201324c193a5a19ffddbe96d4\": container with ID starting with 655be3c00fbbba458f8738dafca93f8b0016812201324c193a5a19ffddbe96d4 not found: ID does not exist" containerID="655be3c00fbbba458f8738dafca93f8b0016812201324c193a5a19ffddbe96d4"
Sep 18 18:18:51.621014 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.620961    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"655be3c00fbbba458f8738dafca93f8b0016812201324c193a5a19ffddbe96d4"} err="failed to get container status \"655be3c00fbbba458f8738dafca93f8b0016812201324c193a5a19ffddbe96d4\": rpc error: code = NotFound desc = could not find container \"655be3c00fbbba458f8738dafca93f8b0016812201324c193a5a19ffddbe96d4\": container with ID starting with 655be3c00fbbba458f8738dafca93f8b0016812201324c193a5a19ffddbe96d4 not found: ID does not exist"
Sep 18 18:18:51.641974 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.641946    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"]
Sep 18 18:18:51.650701 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:51.650677    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-005032-66884b6f5b-94cdq"]
Sep 18 18:18:52.045482 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:52.045452    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58714b51-9041-4005-8496-7c628f8f6c01" path="/var/lib/kubelet/pods/58714b51-9041-4005-8496-7c628f8f6c01/volumes"
Sep 18 18:18:53.394471 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.394402    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z" podUID="432c5685-7f13-45f7-b5a3-44947c7d9c52" containerName="main" containerID="cri-o://6f5563f6d259221247ae7af3be79be76ea8bedccea202e4e3d438caf363f723a" gracePeriod=45
Sep 18 18:18:53.411408 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.411368    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl" podUID="efa321cb-a95c-4b6b-9602-06c67d6aaa72" containerName="main" containerID="cri-o://1ad2731fdefb487fee875df132e9af7c8898ceb937f324f2274f079640167883" gracePeriod=45
Sep 18 18:18:53.518092 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.518053    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl" podUID="efa321cb-a95c-4b6b-9602-06c67d6aaa72" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": dial tcp 10.134.0.60:8000: connect: connection refused"
Sep 18 18:18:53.626893 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.626858    2568 generic.go:356] "Generic (PLEG): container finished" podID="432c5685-7f13-45f7-b5a3-44947c7d9c52" containerID="6f5563f6d259221247ae7af3be79be76ea8bedccea202e4e3d438caf363f723a" exitCode=0
Sep 18 18:18:53.627014 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.626930    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z" event={"ID":"432c5685-7f13-45f7-b5a3-44947c7d9c52","Type":"ContainerDied","Data":"6f5563f6d259221247ae7af3be79be76ea8bedccea202e4e3d438caf363f723a"}
Sep 18 18:18:53.629053 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.629025    2568 generic.go:356] "Generic (PLEG): container finished" podID="efa321cb-a95c-4b6b-9602-06c67d6aaa72" containerID="1ad2731fdefb487fee875df132e9af7c8898ceb937f324f2274f079640167883" exitCode=0
Sep 18 18:18:53.629164 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.629102    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl" event={"ID":"efa321cb-a95c-4b6b-9602-06c67d6aaa72","Type":"ContainerDied","Data":"1ad2731fdefb487fee875df132e9af7c8898ceb937f324f2274f079640167883"}
Sep 18 18:18:53.662535 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.662508    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:18:53.698623 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.698453    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:18:53.787901 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.787863    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/432c5685-7f13-45f7-b5a3-44947c7d9c52-tls-certs\") pod \"432c5685-7f13-45f7-b5a3-44947c7d9c52\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") "
Sep 18 18:18:53.788092 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.787923    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-dshm\") pod \"432c5685-7f13-45f7-b5a3-44947c7d9c52\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") "
Sep 18 18:18:53.788092 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.787952    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-tmp-dir\") pod \"432c5685-7f13-45f7-b5a3-44947c7d9c52\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") "
Sep 18 18:18:53.788092 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.787980    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-dshm\") pod \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") "
Sep 18 18:18:53.788092 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788003    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2x85j\" (UniqueName: \"kubernetes.io/projected/432c5685-7f13-45f7-b5a3-44947c7d9c52-kube-api-access-2x85j\") pod \"432c5685-7f13-45f7-b5a3-44947c7d9c52\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") "
Sep 18 18:18:53.788092 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788031    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-tmp-dir\") pod \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") "
Sep 18 18:18:53.788092 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788054    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2djx\" (UniqueName: \"kubernetes.io/projected/efa321cb-a95c-4b6b-9602-06c67d6aaa72-kube-api-access-w2djx\") pod \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") "
Sep 18 18:18:53.788092 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788075    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-model-cache\") pod \"432c5685-7f13-45f7-b5a3-44947c7d9c52\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") "
Sep 18 18:18:53.788475 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788119    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-home\") pod \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") "
Sep 18 18:18:53.788475 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788146    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa321cb-a95c-4b6b-9602-06c67d6aaa72-tls-certs\") pod \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") "
Sep 18 18:18:53.788475 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788244    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-home\") pod \"432c5685-7f13-45f7-b5a3-44947c7d9c52\" (UID: \"432c5685-7f13-45f7-b5a3-44947c7d9c52\") "
Sep 18 18:18:53.788475 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788247    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "432c5685-7f13-45f7-b5a3-44947c7d9c52" (UID: "432c5685-7f13-45f7-b5a3-44947c7d9c52"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:18:53.788475 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788294    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-model-cache\") pod \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\" (UID: \"efa321cb-a95c-4b6b-9602-06c67d6aaa72\") "
Sep 18 18:18:53.788475 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788428    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-model-cache" (OuterVolumeSpecName: "model-cache") pod "432c5685-7f13-45f7-b5a3-44947c7d9c52" (UID: "432c5685-7f13-45f7-b5a3-44947c7d9c52"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:18:53.788834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788650    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-model-cache" (OuterVolumeSpecName: "model-cache") pod "efa321cb-a95c-4b6b-9602-06c67d6aaa72" (UID: "efa321cb-a95c-4b6b-9602-06c67d6aaa72"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:18:53.788834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788662    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "efa321cb-a95c-4b6b-9602-06c67d6aaa72" (UID: "efa321cb-a95c-4b6b-9602-06c67d6aaa72"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:18:53.788834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788768    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-model-cache\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:53.788834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788831    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-tmp-dir\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:53.789054 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788847    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-tmp-dir\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:53.789054 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788856    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-home" (OuterVolumeSpecName: "home") pod "432c5685-7f13-45f7-b5a3-44947c7d9c52" (UID: "432c5685-7f13-45f7-b5a3-44947c7d9c52"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:18:53.789054 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.788860    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-model-cache\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:53.789054 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.789045    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-home" (OuterVolumeSpecName: "home") pod "efa321cb-a95c-4b6b-9602-06c67d6aaa72" (UID: "efa321cb-a95c-4b6b-9602-06c67d6aaa72"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:18:53.790633 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.790603    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-dshm" (OuterVolumeSpecName: "dshm") pod "432c5685-7f13-45f7-b5a3-44947c7d9c52" (UID: "432c5685-7f13-45f7-b5a3-44947c7d9c52"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:18:53.790762 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.790635    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/432c5685-7f13-45f7-b5a3-44947c7d9c52-kube-api-access-2x85j" (OuterVolumeSpecName: "kube-api-access-2x85j") pod "432c5685-7f13-45f7-b5a3-44947c7d9c52" (UID: "432c5685-7f13-45f7-b5a3-44947c7d9c52"). InnerVolumeSpecName "kube-api-access-2x85j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:18:53.790762 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.790722    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa321cb-a95c-4b6b-9602-06c67d6aaa72-kube-api-access-w2djx" (OuterVolumeSpecName: "kube-api-access-w2djx") pod "efa321cb-a95c-4b6b-9602-06c67d6aaa72" (UID: "efa321cb-a95c-4b6b-9602-06c67d6aaa72"). InnerVolumeSpecName "kube-api-access-w2djx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:18:53.790956 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.790921    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-dshm" (OuterVolumeSpecName: "dshm") pod "efa321cb-a95c-4b6b-9602-06c67d6aaa72" (UID: "efa321cb-a95c-4b6b-9602-06c67d6aaa72"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:18:53.791194 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.791157    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efa321cb-a95c-4b6b-9602-06c67d6aaa72-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "efa321cb-a95c-4b6b-9602-06c67d6aaa72" (UID: "efa321cb-a95c-4b6b-9602-06c67d6aaa72"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:18:53.791409 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.791395    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/432c5685-7f13-45f7-b5a3-44947c7d9c52-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "432c5685-7f13-45f7-b5a3-44947c7d9c52" (UID: "432c5685-7f13-45f7-b5a3-44947c7d9c52"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:18:53.889692 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.889651    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-home\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:53.889692 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.889685    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/432c5685-7f13-45f7-b5a3-44947c7d9c52-tls-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:53.889692 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.889695    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/432c5685-7f13-45f7-b5a3-44947c7d9c52-dshm\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:53.889692 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.889702    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-dshm\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:53.889968 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.889710    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x85j\" (UniqueName: \"kubernetes.io/projected/432c5685-7f13-45f7-b5a3-44947c7d9c52-kube-api-access-2x85j\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:53.889968 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.889720    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2djx\" (UniqueName: \"kubernetes.io/projected/efa321cb-a95c-4b6b-9602-06c67d6aaa72-kube-api-access-w2djx\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:53.889968 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.889728    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efa321cb-a95c-4b6b-9602-06c67d6aaa72-home\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:53.889968 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:53.889737    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa321cb-a95c-4b6b-9602-06c67d6aaa72-tls-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:18:54.635698 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:54.635667    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"
Sep 18 18:18:54.636152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:54.635708    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z" event={"ID":"432c5685-7f13-45f7-b5a3-44947c7d9c52","Type":"ContainerDied","Data":"89763bda4c0c6208dad50d86f6585945d2b141fccdbbac229e8f92d19cb1925d"}
Sep 18 18:18:54.636152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:54.635764    2568 scope.go:117] "RemoveContainer" containerID="6f5563f6d259221247ae7af3be79be76ea8bedccea202e4e3d438caf363f723a"
Sep 18 18:18:54.637210 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:54.637154    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl" event={"ID":"efa321cb-a95c-4b6b-9602-06c67d6aaa72","Type":"ContainerDied","Data":"5fdee25ed2dcaf2e3b72719ddf384a37800042d61450aa2a6c985cabeb06b63b"}
Sep 18 18:18:54.637295 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:54.637189    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"
Sep 18 18:18:54.646966 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:54.646945    2568 scope.go:117] "RemoveContainer" containerID="1ad2731fdefb487fee875df132e9af7c8898ceb937f324f2274f079640167883"
Sep 18 18:18:54.661434 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:54.661373    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"]
Sep 18 18:18:54.679455 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:54.679431    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-03e512/distinct-2-w4-03e512-kserve-7cd64746c6-pjm9z"]
Sep 18 18:18:54.693284 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:54.693263    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"]
Sep 18 18:18:54.698676 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:54.698655    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-03e512/premium-w4-03e512-kserve-ccdf875b-6f8jl"]
Sep 18 18:18:56.046006 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:56.045974    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="432c5685-7f13-45f7-b5a3-44947c7d9c52" path="/var/lib/kubelet/pods/432c5685-7f13-45f7-b5a3-44947c7d9c52/volumes"
Sep 18 18:18:56.046451 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:56.046435    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efa321cb-a95c-4b6b-9602-06c67d6aaa72" path="/var/lib/kubelet/pods/efa321cb-a95c-4b6b-9602-06c67d6aaa72/volumes"
Sep 18 18:18:58.205464 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:58.205434    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"]
Sep 18 18:18:58.205948 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:58.205674    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84" podUID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerName="istio-proxy" containerID="cri-o://37abb756d3eb453dffa69df601f345fe9e053bbf2659b6b16b7056503a206fd8" gracePeriod=30
Sep 18 18:18:58.553768 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:58.553682    2568 patch_prober.go:28] interesting pod/e2e-api-b-005032-openshift-default-6666b966b-nfw84 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 18:18:58.553768 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:58.553738    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84" podUID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:18:59.239319 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:18:59.239284    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:19:03.484556 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.484531    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:19:03.486499 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.486479    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-podinfo\") pod \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") "
Sep 18 18:19:03.486609 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.486512    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xczxb\" (UniqueName: \"kubernetes.io/projected/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-kube-api-access-xczxb\") pod \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") "
Sep 18 18:19:03.486609 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.486545    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istiod-ca-cert\") pod \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") "
Sep 18 18:19:03.486609 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.486564    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-workload-certs\") pod \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") "
Sep 18 18:19:03.486609 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.486582    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-credential-socket\") pod \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") "
Sep 18 18:19:03.486840 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.486631    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-token\") pod \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") "
Sep 18 18:19:03.486840 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.486652    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-envoy\") pod \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") "
Sep 18 18:19:03.486840 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.486704    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-workload-socket\") pod \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") "
Sep 18 18:19:03.486840 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.486731    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-data\") pod \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\" (UID: \"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc\") "
Sep 18 18:19:03.487053 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.486838    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" (UID: "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:19:03.487053 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.486882    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" (UID: "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:19:03.487053 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.486907    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" (UID: "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:19:03.487240 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.487039    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-data" (OuterVolumeSpecName: "istio-data") pod "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" (UID: "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:19:03.487240 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.487125    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" (UID: "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:19:03.487240 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.487210    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:03.487240 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.487227    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:03.487240 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.487240    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:03.487507 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.487253    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:03.488873 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.488838    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" (UID: "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:19:03.488992 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.488907    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" (UID: "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:19:03.489055 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.488987    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-token" (OuterVolumeSpecName: "istio-token") pod "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" (UID: "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:19:03.489055 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.488998    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-kube-api-access-xczxb" (OuterVolumeSpecName: "kube-api-access-xczxb") pod "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" (UID: "861b37c6-902d-48f6-8ffa-9be5cf4eb5cc"). InnerVolumeSpecName "kube-api-access-xczxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:19:03.588218 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.588158    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:03.588218 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.588214    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xczxb\" (UniqueName: \"kubernetes.io/projected/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-kube-api-access-xczxb\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:03.588218 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.588231    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:03.588479 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.588243    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:03.588479 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.588255    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:03.698385 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.698346    2568 generic.go:356] "Generic (PLEG): container finished" podID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerID="37abb756d3eb453dffa69df601f345fe9e053bbf2659b6b16b7056503a206fd8" exitCode=0
Sep 18 18:19:03.698574 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.698413    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"
Sep 18 18:19:03.698574 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.698420    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84" event={"ID":"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc","Type":"ContainerDied","Data":"37abb756d3eb453dffa69df601f345fe9e053bbf2659b6b16b7056503a206fd8"}
Sep 18 18:19:03.698574 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.698495    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84" event={"ID":"861b37c6-902d-48f6-8ffa-9be5cf4eb5cc","Type":"ContainerDied","Data":"18c5638c326c828b1038e1f92d120a30399ed2f223593744bdac36644fd3fc3c"}
Sep 18 18:19:03.698574 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.698517    2568 scope.go:117] "RemoveContainer" containerID="37abb756d3eb453dffa69df601f345fe9e053bbf2659b6b16b7056503a206fd8"
Sep 18 18:19:03.708615 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.708594    2568 scope.go:117] "RemoveContainer" containerID="99916783bcdd3cbb3eb58805d2796c56ace7da60ca9ecbd996fe5fbbe318db8c"
Sep 18 18:19:03.718480 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.718451    2568 scope.go:117] "RemoveContainer" containerID="37abb756d3eb453dffa69df601f345fe9e053bbf2659b6b16b7056503a206fd8"
Sep 18 18:19:03.718810 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:19:03.718792    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37abb756d3eb453dffa69df601f345fe9e053bbf2659b6b16b7056503a206fd8\": container with ID starting with 37abb756d3eb453dffa69df601f345fe9e053bbf2659b6b16b7056503a206fd8 not found: ID does not exist" containerID="37abb756d3eb453dffa69df601f345fe9e053bbf2659b6b16b7056503a206fd8"
Sep 18 18:19:03.718877 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.718819    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37abb756d3eb453dffa69df601f345fe9e053bbf2659b6b16b7056503a206fd8"} err="failed to get container status \"37abb756d3eb453dffa69df601f345fe9e053bbf2659b6b16b7056503a206fd8\": rpc error: code = NotFound desc = could not find container \"37abb756d3eb453dffa69df601f345fe9e053bbf2659b6b16b7056503a206fd8\": container with ID starting with 37abb756d3eb453dffa69df601f345fe9e053bbf2659b6b16b7056503a206fd8 not found: ID does not exist"
Sep 18 18:19:03.718877 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.718839    2568 scope.go:117] "RemoveContainer" containerID="99916783bcdd3cbb3eb58805d2796c56ace7da60ca9ecbd996fe5fbbe318db8c"
Sep 18 18:19:03.719119 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:19:03.719101    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99916783bcdd3cbb3eb58805d2796c56ace7da60ca9ecbd996fe5fbbe318db8c\": container with ID starting with 99916783bcdd3cbb3eb58805d2796c56ace7da60ca9ecbd996fe5fbbe318db8c not found: ID does not exist" containerID="99916783bcdd3cbb3eb58805d2796c56ace7da60ca9ecbd996fe5fbbe318db8c"
Sep 18 18:19:03.719158 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.719127    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99916783bcdd3cbb3eb58805d2796c56ace7da60ca9ecbd996fe5fbbe318db8c"} err="failed to get container status \"99916783bcdd3cbb3eb58805d2796c56ace7da60ca9ecbd996fe5fbbe318db8c\": rpc error: code = NotFound desc = could not find container \"99916783bcdd3cbb3eb58805d2796c56ace7da60ca9ecbd996fe5fbbe318db8c\": container with ID starting with 99916783bcdd3cbb3eb58805d2796c56ace7da60ca9ecbd996fe5fbbe318db8c not found: ID does not exist"
Sep 18 18:19:03.720437 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.720419    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"]
Sep 18 18:19:03.726563 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:03.726539    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-005032-openshift-default-6666b966b-nfw84"]
Sep 18 18:19:04.046587 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:04.046504    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" path="/var/lib/kubelet/pods/861b37c6-902d-48f6-8ffa-9be5cf4eb5cc/volumes"
Sep 18 18:19:08.991965 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:08.991924    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"]
Sep 18 18:19:08.992416 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:08.992336    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh" podUID="738736de-5f4e-468e-9971-60489e044a2f" containerName="maas-api" containerID="cri-o://fd5655cb662738259e72881bceea5c5593371147c0f11f943a5c7331871095bb" gracePeriod=30
Sep 18 18:19:09.246115 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.246061    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:19:09.341927 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.341895    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/738736de-5f4e-468e-9971-60489e044a2f-metrics-certs\") pod \"738736de-5f4e-468e-9971-60489e044a2f\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") "
Sep 18 18:19:09.342122 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.341952    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/738736de-5f4e-468e-9971-60489e044a2f-openshift-service-ca\") pod \"738736de-5f4e-468e-9971-60489e044a2f\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") "
Sep 18 18:19:09.342122 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.341992    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/738736de-5f4e-468e-9971-60489e044a2f-maas-api-tls\") pod \"738736de-5f4e-468e-9971-60489e044a2f\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") "
Sep 18 18:19:09.342122 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.342042    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zt965\" (UniqueName: \"kubernetes.io/projected/738736de-5f4e-468e-9971-60489e044a2f-kube-api-access-zt965\") pod \"738736de-5f4e-468e-9971-60489e044a2f\" (UID: \"738736de-5f4e-468e-9971-60489e044a2f\") "
Sep 18 18:19:09.342438 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.342417    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/738736de-5f4e-468e-9971-60489e044a2f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "738736de-5f4e-468e-9971-60489e044a2f" (UID: "738736de-5f4e-468e-9971-60489e044a2f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:19:09.344271 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.344237    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/738736de-5f4e-468e-9971-60489e044a2f-kube-api-access-zt965" (OuterVolumeSpecName: "kube-api-access-zt965") pod "738736de-5f4e-468e-9971-60489e044a2f" (UID: "738736de-5f4e-468e-9971-60489e044a2f"). InnerVolumeSpecName "kube-api-access-zt965". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:19:09.344271 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.344248    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/738736de-5f4e-468e-9971-60489e044a2f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "738736de-5f4e-468e-9971-60489e044a2f" (UID: "738736de-5f4e-468e-9971-60489e044a2f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:19:09.344454 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.344291    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/738736de-5f4e-468e-9971-60489e044a2f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "738736de-5f4e-468e-9971-60489e044a2f" (UID: "738736de-5f4e-468e-9971-60489e044a2f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:19:09.443663 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.443615    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt965\" (UniqueName: \"kubernetes.io/projected/738736de-5f4e-468e-9971-60489e044a2f-kube-api-access-zt965\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:09.443663 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.443655    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/738736de-5f4e-468e-9971-60489e044a2f-metrics-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:09.443663 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.443669    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/738736de-5f4e-468e-9971-60489e044a2f-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:09.443924 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.443681    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/738736de-5f4e-468e-9971-60489e044a2f-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:09.733255 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.733220    2568 generic.go:356] "Generic (PLEG): container finished" podID="738736de-5f4e-468e-9971-60489e044a2f" containerID="fd5655cb662738259e72881bceea5c5593371147c0f11f943a5c7331871095bb" exitCode=0
Sep 18 18:19:09.733449 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.733311    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"
Sep 18 18:19:09.733449 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.733348    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh" event={"ID":"738736de-5f4e-468e-9971-60489e044a2f","Type":"ContainerDied","Data":"fd5655cb662738259e72881bceea5c5593371147c0f11f943a5c7331871095bb"}
Sep 18 18:19:09.733449 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.733408    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh" event={"ID":"738736de-5f4e-468e-9971-60489e044a2f","Type":"ContainerDied","Data":"b4c60ce1f654fabc0ff84e3f69a22498fe47c34c5582217a07db9afc9c83d403"}
Sep 18 18:19:09.733449 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.733429    2568 scope.go:117] "RemoveContainer" containerID="fd5655cb662738259e72881bceea5c5593371147c0f11f943a5c7331871095bb"
Sep 18 18:19:09.743041 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.743024    2568 scope.go:117] "RemoveContainer" containerID="fd5655cb662738259e72881bceea5c5593371147c0f11f943a5c7331871095bb"
Sep 18 18:19:09.743357 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:19:09.743338    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd5655cb662738259e72881bceea5c5593371147c0f11f943a5c7331871095bb\": container with ID starting with fd5655cb662738259e72881bceea5c5593371147c0f11f943a5c7331871095bb not found: ID does not exist" containerID="fd5655cb662738259e72881bceea5c5593371147c0f11f943a5c7331871095bb"
Sep 18 18:19:09.743420 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.743366    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd5655cb662738259e72881bceea5c5593371147c0f11f943a5c7331871095bb"} err="failed to get container status \"fd5655cb662738259e72881bceea5c5593371147c0f11f943a5c7331871095bb\": rpc error: code = NotFound desc = could not find container \"fd5655cb662738259e72881bceea5c5593371147c0f11f943a5c7331871095bb\": container with ID starting with fd5655cb662738259e72881bceea5c5593371147c0f11f943a5c7331871095bb not found: ID does not exist"
Sep 18 18:19:09.754889 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.754861    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"]
Sep 18 18:19:09.760312 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:09.760285    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-28ddec-8457c5bf9d-nsvmh"]
Sep 18 18:19:10.046584 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:10.046504    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="738736de-5f4e-468e-9971-60489e044a2f" path="/var/lib/kubelet/pods/738736de-5f4e-468e-9971-60489e044a2f/volumes"
Sep 18 18:19:17.303037 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:17.303006    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"]
Sep 18 18:19:17.303457 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:17.303240    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" podUID="4e322580-3bf4-44d3-9c10-54f664524069" containerName="istio-proxy" containerID="cri-o://546d401f56050a801bcb0be5da3c9f43e06be418be304a70080972c09d303576" gracePeriod=30
Sep 18 18:19:18.232507 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:18.232468    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 18:19:18.232668 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:18.232540    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" podUID="4e322580-3bf4-44d3-9c10-54f664524069" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:19:20.761756 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:20.761713    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:19:22.570452 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.570389    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:19:22.681920 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.681890    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdv4n\" (UniqueName: \"kubernetes.io/projected/4e322580-3bf4-44d3-9c10-54f664524069-kube-api-access-mdv4n\") pod \"4e322580-3bf4-44d3-9c10-54f664524069\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") "
Sep 18 18:19:22.682112 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.681945    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-workload-certs\") pod \"4e322580-3bf4-44d3-9c10-54f664524069\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") "
Sep 18 18:19:22.682112 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.681982    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-workload-socket\") pod \"4e322580-3bf4-44d3-9c10-54f664524069\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") "
Sep 18 18:19:22.682112 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.682036    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-istio-data\") pod \"4e322580-3bf4-44d3-9c10-54f664524069\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") "
Sep 18 18:19:22.682112 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.682058    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-credential-socket\") pod \"4e322580-3bf4-44d3-9c10-54f664524069\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") "
Sep 18 18:19:22.682380 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.682122    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e322580-3bf4-44d3-9c10-54f664524069-istio-podinfo\") pod \"4e322580-3bf4-44d3-9c10-54f664524069\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") "
Sep 18 18:19:22.682380 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.682162    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e322580-3bf4-44d3-9c10-54f664524069-istio-token\") pod \"4e322580-3bf4-44d3-9c10-54f664524069\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") "
Sep 18 18:19:22.682380 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.682212    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e322580-3bf4-44d3-9c10-54f664524069-istiod-ca-cert\") pod \"4e322580-3bf4-44d3-9c10-54f664524069\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") "
Sep 18 18:19:22.682380 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.682271    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-istio-envoy\") pod \"4e322580-3bf4-44d3-9c10-54f664524069\" (UID: \"4e322580-3bf4-44d3-9c10-54f664524069\") "
Sep 18 18:19:22.682380 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.682342    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4e322580-3bf4-44d3-9c10-54f664524069" (UID: "4e322580-3bf4-44d3-9c10-54f664524069"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:19:22.682380 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.682356    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4e322580-3bf4-44d3-9c10-54f664524069" (UID: "4e322580-3bf4-44d3-9c10-54f664524069"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:19:22.682671 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.682374    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-istio-data" (OuterVolumeSpecName: "istio-data") pod "4e322580-3bf4-44d3-9c10-54f664524069" (UID: "4e322580-3bf4-44d3-9c10-54f664524069"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:19:22.682671 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.682446    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4e322580-3bf4-44d3-9c10-54f664524069" (UID: "4e322580-3bf4-44d3-9c10-54f664524069"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:19:22.682671 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.682649    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:22.682813 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.682670    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:22.682813 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.682685    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:22.682813 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.682699    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:22.682813 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.682699    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e322580-3bf4-44d3-9c10-54f664524069-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4e322580-3bf4-44d3-9c10-54f664524069" (UID: "4e322580-3bf4-44d3-9c10-54f664524069"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:19:22.684365 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.684341    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4e322580-3bf4-44d3-9c10-54f664524069" (UID: "4e322580-3bf4-44d3-9c10-54f664524069"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:19:22.684471 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.684342    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e322580-3bf4-44d3-9c10-54f664524069-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4e322580-3bf4-44d3-9c10-54f664524069" (UID: "4e322580-3bf4-44d3-9c10-54f664524069"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:19:22.684632 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.684609    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e322580-3bf4-44d3-9c10-54f664524069-kube-api-access-mdv4n" (OuterVolumeSpecName: "kube-api-access-mdv4n") pod "4e322580-3bf4-44d3-9c10-54f664524069" (UID: "4e322580-3bf4-44d3-9c10-54f664524069"). InnerVolumeSpecName "kube-api-access-mdv4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:19:22.684733 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.684633    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e322580-3bf4-44d3-9c10-54f664524069-istio-token" (OuterVolumeSpecName: "istio-token") pod "4e322580-3bf4-44d3-9c10-54f664524069" (UID: "4e322580-3bf4-44d3-9c10-54f664524069"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:19:22.783585 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.783554    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e322580-3bf4-44d3-9c10-54f664524069-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:22.783585 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.783582    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdv4n\" (UniqueName: \"kubernetes.io/projected/4e322580-3bf4-44d3-9c10-54f664524069-kube-api-access-mdv4n\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:22.783585 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.783592    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e322580-3bf4-44d3-9c10-54f664524069-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:22.783816 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.783600    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e322580-3bf4-44d3-9c10-54f664524069-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:22.783816 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.783607    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e322580-3bf4-44d3-9c10-54f664524069-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:19:22.820693 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.820604    2568 generic.go:356] "Generic (PLEG): container finished" podID="4e322580-3bf4-44d3-9c10-54f664524069" containerID="546d401f56050a801bcb0be5da3c9f43e06be418be304a70080972c09d303576" exitCode=0
Sep 18 18:19:22.820693 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.820651    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"
Sep 18 18:19:22.820880 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.820696    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" event={"ID":"4e322580-3bf4-44d3-9c10-54f664524069","Type":"ContainerDied","Data":"546d401f56050a801bcb0be5da3c9f43e06be418be304a70080972c09d303576"}
Sep 18 18:19:22.820880 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.820775    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs" event={"ID":"4e322580-3bf4-44d3-9c10-54f664524069","Type":"ContainerDied","Data":"9428c6fbb3b02d364471f2d613eff369c9b3a0c287969b9af8077cde3d74335c"}
Sep 18 18:19:22.820880 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.820799    2568 scope.go:117] "RemoveContainer" containerID="546d401f56050a801bcb0be5da3c9f43e06be418be304a70080972c09d303576"
Sep 18 18:19:22.830687 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.830670    2568 scope.go:117] "RemoveContainer" containerID="a67725b77e91cca9b532ba79e308676c444c86d199596ef3154208cbcd083552"
Sep 18 18:19:22.838721 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.838703    2568 scope.go:117] "RemoveContainer" containerID="546d401f56050a801bcb0be5da3c9f43e06be418be304a70080972c09d303576"
Sep 18 18:19:22.838988 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:19:22.838969    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"546d401f56050a801bcb0be5da3c9f43e06be418be304a70080972c09d303576\": container with ID starting with 546d401f56050a801bcb0be5da3c9f43e06be418be304a70080972c09d303576 not found: ID does not exist" containerID="546d401f56050a801bcb0be5da3c9f43e06be418be304a70080972c09d303576"
Sep 18 18:19:22.839033 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.838996    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"546d401f56050a801bcb0be5da3c9f43e06be418be304a70080972c09d303576"} err="failed to get container status \"546d401f56050a801bcb0be5da3c9f43e06be418be304a70080972c09d303576\": rpc error: code = NotFound desc = could not find container \"546d401f56050a801bcb0be5da3c9f43e06be418be304a70080972c09d303576\": container with ID starting with 546d401f56050a801bcb0be5da3c9f43e06be418be304a70080972c09d303576 not found: ID does not exist"
Sep 18 18:19:22.839033 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.839014    2568 scope.go:117] "RemoveContainer" containerID="a67725b77e91cca9b532ba79e308676c444c86d199596ef3154208cbcd083552"
Sep 18 18:19:22.839383 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:19:22.839357    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a67725b77e91cca9b532ba79e308676c444c86d199596ef3154208cbcd083552\": container with ID starting with a67725b77e91cca9b532ba79e308676c444c86d199596ef3154208cbcd083552 not found: ID does not exist" containerID="a67725b77e91cca9b532ba79e308676c444c86d199596ef3154208cbcd083552"
Sep 18 18:19:22.839545 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.839388    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a67725b77e91cca9b532ba79e308676c444c86d199596ef3154208cbcd083552"} err="failed to get container status \"a67725b77e91cca9b532ba79e308676c444c86d199596ef3154208cbcd083552\": rpc error: code = NotFound desc = could not find container \"a67725b77e91cca9b532ba79e308676c444c86d199596ef3154208cbcd083552\": container with ID starting with a67725b77e91cca9b532ba79e308676c444c86d199596ef3154208cbcd083552 not found: ID does not exist"
Sep 18 18:19:22.841934 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.841911    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"]
Sep 18 18:19:22.845745 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:22.845725    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-28ddec-openshift-default-798d577b9d-jskgs"]
Sep 18 18:19:24.045235 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:24.045204    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e322580-3bf4-44d3-9c10-54f664524069" path="/var/lib/kubelet/pods/4e322580-3bf4-44d3-9c10-54f664524069/volumes"
Sep 18 18:19:34.628301 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:34.628260    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:19:58.128187 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:58.128143    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 18:19:58.134714 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:19:58.134694    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 18:20:02.324389 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:20:02.324350    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:20:26.214660 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:20:26.214622    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:20:39.522608 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:20:39.522568    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:20:51.626046 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:20:51.626004    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:20:58.718777 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:20:58.718735    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:21:25.367142 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:21:25.367111    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:21:40.315085 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:21:40.315052    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:21:51.215571 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:21:51.215535    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:22:05.715435 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:22:05.715401    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:22:38.526927 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:22:38.526889    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:23:08.919044 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919006    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"]
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919440    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e322580-3bf4-44d3-9c10-54f664524069" containerName="istio-proxy"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919454    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e322580-3bf4-44d3-9c10-54f664524069" containerName="istio-proxy"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919463    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerName="istio-proxy"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919468    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerName="istio-proxy"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919481    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efa321cb-a95c-4b6b-9602-06c67d6aaa72" containerName="main"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919488    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="efa321cb-a95c-4b6b-9602-06c67d6aaa72" containerName="main"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919495    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="432c5685-7f13-45f7-b5a3-44947c7d9c52" containerName="main"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919501    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="432c5685-7f13-45f7-b5a3-44947c7d9c52" containerName="main"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919506    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="028d2288-c88d-45b7-a9b4-8276120ccc19" containerName="maas-api"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919511    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="028d2288-c88d-45b7-a9b4-8276120ccc19" containerName="maas-api"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919526    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="738736de-5f4e-468e-9971-60489e044a2f" containerName="maas-api"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919531    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="738736de-5f4e-468e-9971-60489e044a2f" containerName="maas-api"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919537    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58714b51-9041-4005-8496-7c628f8f6c01" containerName="maas-api"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919542    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="58714b51-9041-4005-8496-7c628f8f6c01" containerName="maas-api"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919603    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="028d2288-c88d-45b7-a9b4-8276120ccc19" containerName="maas-api"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919612    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="738736de-5f4e-468e-9971-60489e044a2f" containerName="maas-api"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919617    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerName="istio-proxy"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919623    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerName="istio-proxy"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919629    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e322580-3bf4-44d3-9c10-54f664524069" containerName="istio-proxy"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919634    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="432c5685-7f13-45f7-b5a3-44947c7d9c52" containerName="main"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919640    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="58714b51-9041-4005-8496-7c628f8f6c01" containerName="maas-api"
Sep 18 18:23:08.921488 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.919646    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="efa321cb-a95c-4b6b-9602-06c67d6aaa72" containerName="main"
Sep 18 18:23:08.922574 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.922560    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:08.926245 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.926223    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-66c663\"/\"ipp-route-66c663-kserve-self-signed-certs\""
Sep 18 18:23:08.926376 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.926279    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-66c663\"/\"openshift-service-ca.crt\""
Sep 18 18:23:08.926376 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.926315    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-66c663\"/\"kube-root-ca.crt\""
Sep 18 18:23:08.926376 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.926289    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-66c663\"/\"default-dockercfg-swkcl\""
Sep 18 18:23:08.932471 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.932450    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"]
Sep 18 18:23:08.946603 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.946577    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-home\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:08.946706 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.946617    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-tls-certs\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:08.946706 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.946669    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-dshm\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:08.946843 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.946714    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-tmp-dir\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:08.946843 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.946736    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-model-cache\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:08.946927 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:08.946844    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnpqq\" (UniqueName: \"kubernetes.io/projected/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-kube-api-access-xnpqq\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:09.048104 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:09.048073    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-model-cache\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:09.048308 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:09.048239    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnpqq\" (UniqueName: \"kubernetes.io/projected/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-kube-api-access-xnpqq\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:09.048397 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:09.048337    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-home\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:09.048458 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:09.048395    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-tls-certs\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:09.048522 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:09.048467    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-dshm\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:09.048522 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:09.048509    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-model-cache\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:09.048653 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:09.048521    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-tmp-dir\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:09.048748 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:09.048731    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-home\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:09.048815 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:09.048797    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-tmp-dir\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:09.050713 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:09.050684    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-dshm\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:09.050945 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:09.050929    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-tls-certs\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:09.056050 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:09.056026    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnpqq\" (UniqueName: \"kubernetes.io/projected/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-kube-api-access-xnpqq\") pod \"ipp-route-66c663-kserve-858849cbb7-7hz8n\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") " pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:09.232312 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:09.232232    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:09.508142 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:09.508115    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"]
Sep 18 18:23:09.509691 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:23:09.509665    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8bb151f4_9d47_4427_bc53_c7d5f0d6c84f.slice/crio-59e3f632c7649faa5a59b38cfa973dade46c05d77271a829be94402bbcd2459b WatchSource:0}: Error finding container 59e3f632c7649faa5a59b38cfa973dade46c05d77271a829be94402bbcd2459b: Status 404 returned error can't find the container with id 59e3f632c7649faa5a59b38cfa973dade46c05d77271a829be94402bbcd2459b
Sep 18 18:23:10.052954 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:10.052920    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n" event={"ID":"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f","Type":"ContainerStarted","Data":"83029b836a619d74f9bba07cfd4adaf1dcafec5090f8ca60e1c991cfba063262"}
Sep 18 18:23:10.053366 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:10.052962    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n" event={"ID":"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f","Type":"ContainerStarted","Data":"59e3f632c7649faa5a59b38cfa973dade46c05d77271a829be94402bbcd2459b"}
Sep 18 18:23:10.071243 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:10.071194    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n" podStartSLOduration=2.071161245 podStartE2EDuration="2.071161245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:23:10.069654133 +0000 UTC m=+1994.147458033" watchObservedRunningTime="2026-09-18 18:23:10.071161245 +0000 UTC m=+1994.148965187"
Sep 18 18:23:13.030773 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:13.030738    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:23:19.233309 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:19.233269    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:19.233720 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:19.233327    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:19.246242 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:19.246217    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:20.120400 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:20.120372    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:23:30.226821 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:30.226785    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:23:37.232249 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.232211    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"]
Sep 18 18:23:37.232676 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.232657    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e322580-3bf4-44d3-9c10-54f664524069" containerName="istio-proxy"
Sep 18 18:23:37.232726 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.232677    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e322580-3bf4-44d3-9c10-54f664524069" containerName="istio-proxy"
Sep 18 18:23:37.232726 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.232687    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerName="istio-proxy"
Sep 18 18:23:37.232726 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.232693    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="861b37c6-902d-48f6-8ffa-9be5cf4eb5cc" containerName="istio-proxy"
Sep 18 18:23:37.232821 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.232778    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e322580-3bf4-44d3-9c10-54f664524069" containerName="istio-proxy"
Sep 18 18:23:37.236034 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.236007    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.238524 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.238503    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-f3eedc-openshift-default-dockercfg-65vxp\""
Sep 18 18:23:37.247578 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.247554    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"]
Sep 18 18:23:37.326694 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.326584    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-envoy\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.326694 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.326632    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-podinfo\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.326694 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.326663    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-token\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.327005 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.326712    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-credential-socket\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.327005 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.326749    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-data\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.327005 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.326845    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh66s\" (UniqueName: \"kubernetes.io/projected/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-kube-api-access-mh66s\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.327005 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.326935    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-workload-socket\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.327005 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.326968    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istiod-ca-cert\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.327301 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.327040    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-workload-certs\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.427916 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.427880    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-workload-socket\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.427916 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.427916    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istiod-ca-cert\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.428146 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.427955    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-workload-certs\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.428146 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.427995    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-envoy\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.428286 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.428140    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-podinfo\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.428286 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.428210    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-token\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.428286 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.428271    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-credential-socket\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.428446 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.428286    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-workload-socket\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.428446 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.428330    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-data\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.428446 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.428368    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh66s\" (UniqueName: \"kubernetes.io/projected/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-kube-api-access-mh66s\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.428446 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.428332    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-workload-certs\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.428624 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.428606    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-data\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.428829 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.428806    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istiod-ca-cert\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.428878 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.428812    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-credential-socket\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.430330 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.430309    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-envoy\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.430853 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.430834    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-podinfo\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.436574 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.436549    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh66s\" (UniqueName: \"kubernetes.io/projected/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-kube-api-access-mh66s\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.437023 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.437006    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-token\") pod \"e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") " pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.550233 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.550202    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:37.826429 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.826405    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"]
Sep 18 18:23:37.829375 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:23:37.829344    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54e6baeb_e39c_45dd_9b49_6cf09eff3dbe.slice/crio-9094ca71c643a94f781aee32bdc2a59424288407509122a624fd2614eb033478 WatchSource:0}: Error finding container 9094ca71c643a94f781aee32bdc2a59424288407509122a624fd2614eb033478: Status 404 returned error can't find the container with id 9094ca71c643a94f781aee32bdc2a59424288407509122a624fd2614eb033478
Sep 18 18:23:37.831625 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.831588    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:23:37.831721 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.831661    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:23:37.831721 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:37.831702    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:23:38.213493 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:38.213454    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7" event={"ID":"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe","Type":"ContainerStarted","Data":"b19410176fd2765266cdf0740b39bc93a7a6d5086e88826529494760dd1c3708"}
Sep 18 18:23:38.213677 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:38.213506    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7" event={"ID":"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe","Type":"ContainerStarted","Data":"9094ca71c643a94f781aee32bdc2a59424288407509122a624fd2614eb033478"}
Sep 18 18:23:38.233813 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:38.233767    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7" podStartSLOduration=1.23375168 podStartE2EDuration="1.23375168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:23:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:23:38.231841551 +0000 UTC m=+2022.309645451" watchObservedRunningTime="2026-09-18 18:23:38.23375168 +0000 UTC m=+2022.311555579"
Sep 18 18:23:38.550869 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:38.550788    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:38.556345 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:38.556319    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:39.218927 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:39.218887    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:39.219928 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:39.219908    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:23:43.396824 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.396795    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"]
Sep 18 18:23:43.400710 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.400689    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:23:43.402998 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.402975    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-f3eedc-serving-cert\""
Sep 18 18:23:43.411594 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.411569    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"]
Sep 18 18:23:43.590206 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.590142    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d66e3772-dd2d-4d90-8014-3aca2be939bc-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:23:43.590386 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.590214    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwh28\" (UniqueName: \"kubernetes.io/projected/d66e3772-dd2d-4d90-8014-3aca2be939bc-kube-api-access-mwh28\") pod \"maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:23:43.590386 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.590242    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d66e3772-dd2d-4d90-8014-3aca2be939bc-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:23:43.590386 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.590329    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d66e3772-dd2d-4d90-8014-3aca2be939bc-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:23:43.691839 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.691801    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d66e3772-dd2d-4d90-8014-3aca2be939bc-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:23:43.692021 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.691901    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d66e3772-dd2d-4d90-8014-3aca2be939bc-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:23:43.692021 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.691930    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwh28\" (UniqueName: \"kubernetes.io/projected/d66e3772-dd2d-4d90-8014-3aca2be939bc-kube-api-access-mwh28\") pod \"maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:23:43.692021 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.691948    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d66e3772-dd2d-4d90-8014-3aca2be939bc-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:23:43.692705 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.692679    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d66e3772-dd2d-4d90-8014-3aca2be939bc-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:23:43.694478 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.694451    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d66e3772-dd2d-4d90-8014-3aca2be939bc-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:23:43.694770 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.694750    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d66e3772-dd2d-4d90-8014-3aca2be939bc-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:23:43.712051 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.712018    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwh28\" (UniqueName: \"kubernetes.io/projected/d66e3772-dd2d-4d90-8014-3aca2be939bc-kube-api-access-mwh28\") pod \"maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:23:43.713646 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.713621    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:23:43.989584 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.989554    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"]
Sep 18 18:23:43.991646 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:23:43.991613    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66e3772_dd2d_4d90_8014_3aca2be939bc.slice/crio-d3354284d0731a09ef0c1ed94f421b5606eb00ca50643750faca0d32d43d5415 WatchSource:0}: Error finding container d3354284d0731a09ef0c1ed94f421b5606eb00ca50643750faca0d32d43d5415: Status 404 returned error can't find the container with id d3354284d0731a09ef0c1ed94f421b5606eb00ca50643750faca0d32d43d5415
Sep 18 18:23:43.993068 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:43.993052    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:23:44.247548 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:44.247461    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j" event={"ID":"d66e3772-dd2d-4d90-8014-3aca2be939bc","Type":"ContainerStarted","Data":"d3354284d0731a09ef0c1ed94f421b5606eb00ca50643750faca0d32d43d5415"}
Sep 18 18:23:45.254693 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:45.254657    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j" event={"ID":"d66e3772-dd2d-4d90-8014-3aca2be939bc","Type":"ContainerStarted","Data":"8f9bcb2226505e2503a0f1df20fd6b2ae31ee2058eb1e3fbf0715136e2599bea"}
Sep 18 18:23:45.255063 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:45.254717    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:23:45.273508 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:45.273462    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j" podStartSLOduration=1.8689409970000002 podStartE2EDuration="2.273448191s" totalImagesPullingTime="404.507194ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:23:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:23:45.271766289 +0000 UTC m=+2029.349570189" watchObservedRunningTime="2026-09-18 18:23:45.273448191 +0000 UTC m=+2029.351252089"
Sep 18 18:23:49.517302 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:49.517257    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:23:51.265870 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:23:51.265840    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:24:00.339817 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:00.339782    2568 generic.go:356] "Generic (PLEG): container finished" podID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" containerID="b19410176fd2765266cdf0740b39bc93a7a6d5086e88826529494760dd1c3708" exitCode=0
Sep 18 18:24:00.340251 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:00.339841    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7" event={"ID":"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe","Type":"ContainerDied","Data":"b19410176fd2765266cdf0740b39bc93a7a6d5086e88826529494760dd1c3708"}
Sep 18 18:24:00.340251 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:00.340224    2568 scope.go:117] "RemoveContainer" containerID="b19410176fd2765266cdf0740b39bc93a7a6d5086e88826529494760dd1c3708"
Sep 18 18:24:00.341425 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:00.341396    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:24:00.341500 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:00.341463    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:24:00.341500 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:00.341493    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 18 18:24:01.347163 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:01.347125    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7" event={"ID":"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe","Type":"ContainerStarted","Data":"460cc23c4c661aa4ec115f02bdb7ac21a26676b7ca5f49e45ad2644ce249e3cf"}
Sep 18 18:24:01.550510 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:01.550474    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:24:01.550743 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:01.550718    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 18 18:24:01.550840 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:01.550779    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7" podUID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 18 18:24:02.550885 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:02.550846    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 18 18:24:02.551340 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:02.550905    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7" podUID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 18 18:24:03.550776 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:03.550739    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 18 18:24:03.550970 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:03.550804    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7" podUID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 18 18:24:04.550581 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:04.550545    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 18 18:24:04.550762 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:04.550597    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7" podUID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 18 18:24:05.551091 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:05.551057    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 18 18:24:05.551496 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:05.551120    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7" podUID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 18 18:24:06.551385 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:06.551349    2568 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 18 18:24:06.551792 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:06.551406    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7" podUID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 18 18:24:07.554593 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:07.554562    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:24:07.554978 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:07.554905    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:24:07.555558 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:07.555539    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:24:22.057989 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.057905    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"]
Sep 18 18:24:22.058632 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.058140    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j" podUID="d66e3772-dd2d-4d90-8014-3aca2be939bc" containerName="maas-api" containerID="cri-o://8f9bcb2226505e2503a0f1df20fd6b2ae31ee2058eb1e3fbf0715136e2599bea" gracePeriod=30
Sep 18 18:24:22.309484 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.309417    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:24:22.384716 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.384684    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d66e3772-dd2d-4d90-8014-3aca2be939bc-maas-api-tls\") pod \"d66e3772-dd2d-4d90-8014-3aca2be939bc\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") "
Sep 18 18:24:22.384909 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.384746    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d66e3772-dd2d-4d90-8014-3aca2be939bc-metrics-certs\") pod \"d66e3772-dd2d-4d90-8014-3aca2be939bc\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") "
Sep 18 18:24:22.384909 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.384778    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwh28\" (UniqueName: \"kubernetes.io/projected/d66e3772-dd2d-4d90-8014-3aca2be939bc-kube-api-access-mwh28\") pod \"d66e3772-dd2d-4d90-8014-3aca2be939bc\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") "
Sep 18 18:24:22.384909 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.384820    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d66e3772-dd2d-4d90-8014-3aca2be939bc-openshift-service-ca\") pod \"d66e3772-dd2d-4d90-8014-3aca2be939bc\" (UID: \"d66e3772-dd2d-4d90-8014-3aca2be939bc\") "
Sep 18 18:24:22.385259 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.385232    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d66e3772-dd2d-4d90-8014-3aca2be939bc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d66e3772-dd2d-4d90-8014-3aca2be939bc" (UID: "d66e3772-dd2d-4d90-8014-3aca2be939bc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:24:22.386895 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.386870    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d66e3772-dd2d-4d90-8014-3aca2be939bc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d66e3772-dd2d-4d90-8014-3aca2be939bc" (UID: "d66e3772-dd2d-4d90-8014-3aca2be939bc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:24:22.387011 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.386906    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d66e3772-dd2d-4d90-8014-3aca2be939bc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d66e3772-dd2d-4d90-8014-3aca2be939bc" (UID: "d66e3772-dd2d-4d90-8014-3aca2be939bc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:24:22.387011 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.386953    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d66e3772-dd2d-4d90-8014-3aca2be939bc-kube-api-access-mwh28" (OuterVolumeSpecName: "kube-api-access-mwh28") pod "d66e3772-dd2d-4d90-8014-3aca2be939bc" (UID: "d66e3772-dd2d-4d90-8014-3aca2be939bc"). InnerVolumeSpecName "kube-api-access-mwh28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:24:22.475688 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.475649    2568 generic.go:356] "Generic (PLEG): container finished" podID="d66e3772-dd2d-4d90-8014-3aca2be939bc" containerID="8f9bcb2226505e2503a0f1df20fd6b2ae31ee2058eb1e3fbf0715136e2599bea" exitCode=0
Sep 18 18:24:22.475859 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.475696    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"
Sep 18 18:24:22.475859 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.475725    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j" event={"ID":"d66e3772-dd2d-4d90-8014-3aca2be939bc","Type":"ContainerDied","Data":"8f9bcb2226505e2503a0f1df20fd6b2ae31ee2058eb1e3fbf0715136e2599bea"}
Sep 18 18:24:22.475859 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.475779    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j" event={"ID":"d66e3772-dd2d-4d90-8014-3aca2be939bc","Type":"ContainerDied","Data":"d3354284d0731a09ef0c1ed94f421b5606eb00ca50643750faca0d32d43d5415"}
Sep 18 18:24:22.475859 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.475803    2568 scope.go:117] "RemoveContainer" containerID="8f9bcb2226505e2503a0f1df20fd6b2ae31ee2058eb1e3fbf0715136e2599bea"
Sep 18 18:24:22.485694 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.485669    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d66e3772-dd2d-4d90-8014-3aca2be939bc-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:24:22.485694 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.485693    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d66e3772-dd2d-4d90-8014-3aca2be939bc-metrics-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:24:22.485873 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.485702    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwh28\" (UniqueName: \"kubernetes.io/projected/d66e3772-dd2d-4d90-8014-3aca2be939bc-kube-api-access-mwh28\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:24:22.485873 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.485711    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d66e3772-dd2d-4d90-8014-3aca2be939bc-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:24:22.486969 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.486951    2568 scope.go:117] "RemoveContainer" containerID="8f9bcb2226505e2503a0f1df20fd6b2ae31ee2058eb1e3fbf0715136e2599bea"
Sep 18 18:24:22.487299 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:24:22.487278    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f9bcb2226505e2503a0f1df20fd6b2ae31ee2058eb1e3fbf0715136e2599bea\": container with ID starting with 8f9bcb2226505e2503a0f1df20fd6b2ae31ee2058eb1e3fbf0715136e2599bea not found: ID does not exist" containerID="8f9bcb2226505e2503a0f1df20fd6b2ae31ee2058eb1e3fbf0715136e2599bea"
Sep 18 18:24:22.487377 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.487313    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f9bcb2226505e2503a0f1df20fd6b2ae31ee2058eb1e3fbf0715136e2599bea"} err="failed to get container status \"8f9bcb2226505e2503a0f1df20fd6b2ae31ee2058eb1e3fbf0715136e2599bea\": rpc error: code = NotFound desc = could not find container \"8f9bcb2226505e2503a0f1df20fd6b2ae31ee2058eb1e3fbf0715136e2599bea\": container with ID starting with 8f9bcb2226505e2503a0f1df20fd6b2ae31ee2058eb1e3fbf0715136e2599bea not found: ID does not exist"
Sep 18 18:24:22.498382 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.498352    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"]
Sep 18 18:24:22.502567 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:22.502547    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f3eedc-5cc957bffc-wkz8j"]
Sep 18 18:24:24.045816 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:24.045782    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d66e3772-dd2d-4d90-8014-3aca2be939bc" path="/var/lib/kubelet/pods/d66e3772-dd2d-4d90-8014-3aca2be939bc/volumes"
Sep 18 18:24:26.614555 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:26.614516    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:24:28.846155 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:28.846119    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"]
Sep 18 18:24:28.846593 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:28.846364    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7" podUID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" containerName="istio-proxy" containerID="cri-o://460cc23c4c661aa4ec115f02bdb7ac21a26676b7ca5f49e45ad2644ce249e3cf" gracePeriod=30
Sep 18 18:24:34.106865 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.106836    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:24:34.197465 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.197429    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mh66s\" (UniqueName: \"kubernetes.io/projected/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-kube-api-access-mh66s\") pod \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") "
Sep 18 18:24:34.197675 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.197484    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-workload-certs\") pod \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") "
Sep 18 18:24:34.197675 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.197518    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-workload-socket\") pod \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") "
Sep 18 18:24:34.197675 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.197618    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-podinfo\") pod \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") "
Sep 18 18:24:34.197675 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.197667    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-envoy\") pod \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") "
Sep 18 18:24:34.197907 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.197694    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-credential-socket\") pod \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") "
Sep 18 18:24:34.197907 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.197755    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-token\") pod \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") "
Sep 18 18:24:34.197907 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.197781    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-data\") pod \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") "
Sep 18 18:24:34.197907 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.197808    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istiod-ca-cert\") pod \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\" (UID: \"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe\") "
Sep 18 18:24:34.197907 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.197838    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" (UID: "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:34.197907 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.197849    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" (UID: "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:34.198268 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.197982    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" (UID: "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:34.198268 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.198084    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-data" (OuterVolumeSpecName: "istio-data") pod "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" (UID: "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:34.198386 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.198282    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:24:34.198386 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.198303    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:24:34.198386 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.198318    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:24:34.198386 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.198333    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:24:34.198386 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.198353    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" (UID: "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:24:34.199848 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.199820    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" (UID: "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:24:34.199942 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.199908    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-kube-api-access-mh66s" (OuterVolumeSpecName: "kube-api-access-mh66s") pod "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" (UID: "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe"). InnerVolumeSpecName "kube-api-access-mh66s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:24:34.199942 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.199932    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-token" (OuterVolumeSpecName: "istio-token") pod "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" (UID: "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:24:34.200052 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.199974    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" (UID: "54e6baeb-e39c-45dd-9b49-6cf09eff3dbe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:34.299662 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.299627    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:24:34.299662 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.299655    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mh66s\" (UniqueName: \"kubernetes.io/projected/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-kube-api-access-mh66s\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:24:34.299662 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.299666    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:24:34.299906 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.299677    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:24:34.299906 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.299685    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:24:34.545632 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.545542    2568 generic.go:356] "Generic (PLEG): container finished" podID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" containerID="460cc23c4c661aa4ec115f02bdb7ac21a26676b7ca5f49e45ad2644ce249e3cf" exitCode=0
Sep 18 18:24:34.545632 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.545590    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"
Sep 18 18:24:34.545834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.545626    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7" event={"ID":"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe","Type":"ContainerDied","Data":"460cc23c4c661aa4ec115f02bdb7ac21a26676b7ca5f49e45ad2644ce249e3cf"}
Sep 18 18:24:34.545834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.545683    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7" event={"ID":"54e6baeb-e39c-45dd-9b49-6cf09eff3dbe","Type":"ContainerDied","Data":"9094ca71c643a94f781aee32bdc2a59424288407509122a624fd2614eb033478"}
Sep 18 18:24:34.545834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.545708    2568 scope.go:117] "RemoveContainer" containerID="460cc23c4c661aa4ec115f02bdb7ac21a26676b7ca5f49e45ad2644ce249e3cf"
Sep 18 18:24:34.555679 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.555665    2568 scope.go:117] "RemoveContainer" containerID="b19410176fd2765266cdf0740b39bc93a7a6d5086e88826529494760dd1c3708"
Sep 18 18:24:34.565831 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.565810    2568 scope.go:117] "RemoveContainer" containerID="460cc23c4c661aa4ec115f02bdb7ac21a26676b7ca5f49e45ad2644ce249e3cf"
Sep 18 18:24:34.566199 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:24:34.566144    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"460cc23c4c661aa4ec115f02bdb7ac21a26676b7ca5f49e45ad2644ce249e3cf\": container with ID starting with 460cc23c4c661aa4ec115f02bdb7ac21a26676b7ca5f49e45ad2644ce249e3cf not found: ID does not exist" containerID="460cc23c4c661aa4ec115f02bdb7ac21a26676b7ca5f49e45ad2644ce249e3cf"
Sep 18 18:24:34.566295 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.566208    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"460cc23c4c661aa4ec115f02bdb7ac21a26676b7ca5f49e45ad2644ce249e3cf"} err="failed to get container status \"460cc23c4c661aa4ec115f02bdb7ac21a26676b7ca5f49e45ad2644ce249e3cf\": rpc error: code = NotFound desc = could not find container \"460cc23c4c661aa4ec115f02bdb7ac21a26676b7ca5f49e45ad2644ce249e3cf\": container with ID starting with 460cc23c4c661aa4ec115f02bdb7ac21a26676b7ca5f49e45ad2644ce249e3cf not found: ID does not exist"
Sep 18 18:24:34.566295 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.566229    2568 scope.go:117] "RemoveContainer" containerID="b19410176fd2765266cdf0740b39bc93a7a6d5086e88826529494760dd1c3708"
Sep 18 18:24:34.567170 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:24:34.567129    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b19410176fd2765266cdf0740b39bc93a7a6d5086e88826529494760dd1c3708\": container with ID starting with b19410176fd2765266cdf0740b39bc93a7a6d5086e88826529494760dd1c3708 not found: ID does not exist" containerID="b19410176fd2765266cdf0740b39bc93a7a6d5086e88826529494760dd1c3708"
Sep 18 18:24:34.567170 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.567161    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b19410176fd2765266cdf0740b39bc93a7a6d5086e88826529494760dd1c3708"} err="failed to get container status \"b19410176fd2765266cdf0740b39bc93a7a6d5086e88826529494760dd1c3708\": rpc error: code = NotFound desc = could not find container \"b19410176fd2765266cdf0740b39bc93a7a6d5086e88826529494760dd1c3708\": container with ID starting with b19410176fd2765266cdf0740b39bc93a7a6d5086e88826529494760dd1c3708 not found: ID does not exist"
Sep 18 18:24:34.568514 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.568483    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"]
Sep 18 18:24:34.570574 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:34.570550    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f3eedc-openshift-default-57bf6cc6d8-nqzq7"]
Sep 18 18:24:36.045500 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:36.045464    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" path="/var/lib/kubelet/pods/54e6baeb-e39c-45dd-9b49-6cf09eff3dbe/volumes"
Sep 18 18:24:45.517721 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:45.517686    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:24:53.382061 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:53.382025    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"]
Sep 18 18:24:58.162475 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:58.162447    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 18:24:58.171669 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:58.171646    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 18:24:59.014793 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:24:59.014756    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:25:08.401817 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.401773    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n" podUID="8bb151f4-9d47-4427-bc53-c7d5f0d6c84f" containerName="main" containerID="cri-o://83029b836a619d74f9bba07cfd4adaf1dcafec5090f8ca60e1c991cfba063262" gracePeriod=45
Sep 18 18:25:08.662131 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.662103    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:25:08.739132 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.739101    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnpqq\" (UniqueName: \"kubernetes.io/projected/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-kube-api-access-xnpqq\") pod \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") "
Sep 18 18:25:08.739338 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.739140    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-home\") pod \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") "
Sep 18 18:25:08.739338 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.739205    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-tmp-dir\") pod \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") "
Sep 18 18:25:08.739338 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.739227    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-tls-certs\") pod \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") "
Sep 18 18:25:08.739338 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.739250    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-dshm\") pod \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") "
Sep 18 18:25:08.739338 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.739322    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-model-cache\") pod \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\" (UID: \"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f\") "
Sep 18 18:25:08.739582 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.739468    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8bb151f4-9d47-4427-bc53-c7d5f0d6c84f" (UID: "8bb151f4-9d47-4427-bc53-c7d5f0d6c84f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:25:08.739582 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.739539    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-home" (OuterVolumeSpecName: "home") pod "8bb151f4-9d47-4427-bc53-c7d5f0d6c84f" (UID: "8bb151f4-9d47-4427-bc53-c7d5f0d6c84f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:25:08.739698 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.739679    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-tmp-dir\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:25:08.739698 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.739682    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-model-cache" (OuterVolumeSpecName: "model-cache") pod "8bb151f4-9d47-4427-bc53-c7d5f0d6c84f" (UID: "8bb151f4-9d47-4427-bc53-c7d5f0d6c84f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:25:08.739786 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.739698    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-home\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:25:08.741495 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.741471    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-dshm" (OuterVolumeSpecName: "dshm") pod "8bb151f4-9d47-4427-bc53-c7d5f0d6c84f" (UID: "8bb151f4-9d47-4427-bc53-c7d5f0d6c84f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:25:08.741495 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.741480    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8bb151f4-9d47-4427-bc53-c7d5f0d6c84f" (UID: "8bb151f4-9d47-4427-bc53-c7d5f0d6c84f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:25:08.741678 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.741622    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-kube-api-access-xnpqq" (OuterVolumeSpecName: "kube-api-access-xnpqq") pod "8bb151f4-9d47-4427-bc53-c7d5f0d6c84f" (UID: "8bb151f4-9d47-4427-bc53-c7d5f0d6c84f"). InnerVolumeSpecName "kube-api-access-xnpqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:25:08.746882 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.746852    2568 generic.go:356] "Generic (PLEG): container finished" podID="8bb151f4-9d47-4427-bc53-c7d5f0d6c84f" containerID="83029b836a619d74f9bba07cfd4adaf1dcafec5090f8ca60e1c991cfba063262" exitCode=0
Sep 18 18:25:08.746986 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.746897    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"
Sep 18 18:25:08.746986 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.746926    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n" event={"ID":"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f","Type":"ContainerDied","Data":"83029b836a619d74f9bba07cfd4adaf1dcafec5090f8ca60e1c991cfba063262"}
Sep 18 18:25:08.746986 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.746977    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n" event={"ID":"8bb151f4-9d47-4427-bc53-c7d5f0d6c84f","Type":"ContainerDied","Data":"59e3f632c7649faa5a59b38cfa973dade46c05d77271a829be94402bbcd2459b"}
Sep 18 18:25:08.747105 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.746997    2568 scope.go:117] "RemoveContainer" containerID="83029b836a619d74f9bba07cfd4adaf1dcafec5090f8ca60e1c991cfba063262"
Sep 18 18:25:08.762023 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.762001    2568 scope.go:117] "RemoveContainer" containerID="83029b836a619d74f9bba07cfd4adaf1dcafec5090f8ca60e1c991cfba063262"
Sep 18 18:25:08.762334 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:25:08.762313    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83029b836a619d74f9bba07cfd4adaf1dcafec5090f8ca60e1c991cfba063262\": container with ID starting with 83029b836a619d74f9bba07cfd4adaf1dcafec5090f8ca60e1c991cfba063262 not found: ID does not exist" containerID="83029b836a619d74f9bba07cfd4adaf1dcafec5090f8ca60e1c991cfba063262"
Sep 18 18:25:08.762404 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.762343    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83029b836a619d74f9bba07cfd4adaf1dcafec5090f8ca60e1c991cfba063262"} err="failed to get container status \"83029b836a619d74f9bba07cfd4adaf1dcafec5090f8ca60e1c991cfba063262\": rpc error: code = NotFound desc = could not find container \"83029b836a619d74f9bba07cfd4adaf1dcafec5090f8ca60e1c991cfba063262\": container with ID starting with 83029b836a619d74f9bba07cfd4adaf1dcafec5090f8ca60e1c991cfba063262 not found: ID does not exist"
Sep 18 18:25:08.773644 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.773616    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"]
Sep 18 18:25:08.778332 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.778309    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-66c663/ipp-route-66c663-kserve-858849cbb7-7hz8n"]
Sep 18 18:25:08.840545 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.840485    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-tls-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:25:08.840545 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.840530    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-dshm\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:25:08.840545 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.840539    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-model-cache\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:25:08.840545 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:08.840548    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnpqq\" (UniqueName: \"kubernetes.io/projected/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f-kube-api-access-xnpqq\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:25:10.045843 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:10.045809    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bb151f4-9d47-4427-bc53-c7d5f0d6c84f" path="/var/lib/kubelet/pods/8bb151f4-9d47-4427-bc53-c7d5f0d6c84f/volumes"
Sep 18 18:25:50.820359 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:25:50.820282    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:26:05.337995 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:05.337959    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-53cde3-29829255-6g587"]
Sep 18 18:26:05.342010 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:05.341980    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-53cde3-29829255-6g587"]
Sep 18 18:26:06.045401 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:06.045368    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9c995b0-2e3f-4661-9b8e-8a8efb2dd567" path="/var/lib/kubelet/pods/c9c995b0-2e3f-4661-9b8e-8a8efb2dd567/volumes"
Sep 18 18:26:08.212391 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:08.212357    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:26:12.083930 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:12.083892    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"]
Sep 18 18:26:12.084394 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:12.084140    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm" podUID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerName="istio-proxy" containerID="cri-o://1b33f07f4a0f76acb47f16eb83929cd61dad3c6c4db354cb563d02ea6fba328b" gracePeriod=30
Sep 18 18:26:17.353470 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.353412    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:26:17.505435 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.505401    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-envoy\") pod \"4e083287-05ce-4370-b6d9-eb248cb8af8f\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") "
Sep 18 18:26:17.505611 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.505444    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-token\") pod \"4e083287-05ce-4370-b6d9-eb248cb8af8f\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") "
Sep 18 18:26:17.505611 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.505516    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-podinfo\") pod \"4e083287-05ce-4370-b6d9-eb248cb8af8f\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") "
Sep 18 18:26:17.505611 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.505563    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e083287-05ce-4370-b6d9-eb248cb8af8f-istiod-ca-cert\") pod \"4e083287-05ce-4370-b6d9-eb248cb8af8f\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") "
Sep 18 18:26:17.505611 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.505595    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-data\") pod \"4e083287-05ce-4370-b6d9-eb248cb8af8f\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") "
Sep 18 18:26:17.505819 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.505626    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8gnw\" (UniqueName: \"kubernetes.io/projected/4e083287-05ce-4370-b6d9-eb248cb8af8f-kube-api-access-w8gnw\") pod \"4e083287-05ce-4370-b6d9-eb248cb8af8f\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") "
Sep 18 18:26:17.505819 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.505665    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-credential-socket\") pod \"4e083287-05ce-4370-b6d9-eb248cb8af8f\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") "
Sep 18 18:26:17.505819 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.505690    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-workload-certs\") pod \"4e083287-05ce-4370-b6d9-eb248cb8af8f\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") "
Sep 18 18:26:17.505983 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.505888    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-data" (OuterVolumeSpecName: "istio-data") pod "4e083287-05ce-4370-b6d9-eb248cb8af8f" (UID: "4e083287-05ce-4370-b6d9-eb248cb8af8f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:17.505983 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.505932    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-workload-socket\") pod \"4e083287-05ce-4370-b6d9-eb248cb8af8f\" (UID: \"4e083287-05ce-4370-b6d9-eb248cb8af8f\") "
Sep 18 18:26:17.505983 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.505959    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e083287-05ce-4370-b6d9-eb248cb8af8f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4e083287-05ce-4370-b6d9-eb248cb8af8f" (UID: "4e083287-05ce-4370-b6d9-eb248cb8af8f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:26:17.506134 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.505988    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4e083287-05ce-4370-b6d9-eb248cb8af8f" (UID: "4e083287-05ce-4370-b6d9-eb248cb8af8f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:17.506372 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.506344    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4e083287-05ce-4370-b6d9-eb248cb8af8f" (UID: "4e083287-05ce-4370-b6d9-eb248cb8af8f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:17.506499 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.506404    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4e083287-05ce-4370-b6d9-eb248cb8af8f" (UID: "4e083287-05ce-4370-b6d9-eb248cb8af8f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:17.506499 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.506425    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e083287-05ce-4370-b6d9-eb248cb8af8f-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:17.506499 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.506448    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:17.506499 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.506462    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:17.506499 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.506475    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:17.507795 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.507766    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4e083287-05ce-4370-b6d9-eb248cb8af8f" (UID: "4e083287-05ce-4370-b6d9-eb248cb8af8f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:26:17.507897 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.507828    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4e083287-05ce-4370-b6d9-eb248cb8af8f" (UID: "4e083287-05ce-4370-b6d9-eb248cb8af8f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:17.507897 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.507835    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-token" (OuterVolumeSpecName: "istio-token") pod "4e083287-05ce-4370-b6d9-eb248cb8af8f" (UID: "4e083287-05ce-4370-b6d9-eb248cb8af8f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:17.507981 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.507890    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e083287-05ce-4370-b6d9-eb248cb8af8f-kube-api-access-w8gnw" (OuterVolumeSpecName: "kube-api-access-w8gnw") pod "4e083287-05ce-4370-b6d9-eb248cb8af8f" (UID: "4e083287-05ce-4370-b6d9-eb248cb8af8f"). InnerVolumeSpecName "kube-api-access-w8gnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:17.607880 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.607825    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:17.607880 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.607846    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8gnw\" (UniqueName: \"kubernetes.io/projected/4e083287-05ce-4370-b6d9-eb248cb8af8f-kube-api-access-w8gnw\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:17.607880 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.607854    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:17.607880 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.607863    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:17.607880 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:17.607873    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e083287-05ce-4370-b6d9-eb248cb8af8f-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:18.122642 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:18.122612    2568 generic.go:356] "Generic (PLEG): container finished" podID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerID="1b33f07f4a0f76acb47f16eb83929cd61dad3c6c4db354cb563d02ea6fba328b" exitCode=0
Sep 18 18:26:18.122828 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:18.122663    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"
Sep 18 18:26:18.122828 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:18.122698    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm" event={"ID":"4e083287-05ce-4370-b6d9-eb248cb8af8f","Type":"ContainerDied","Data":"1b33f07f4a0f76acb47f16eb83929cd61dad3c6c4db354cb563d02ea6fba328b"}
Sep 18 18:26:18.122828 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:18.122756    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm" event={"ID":"4e083287-05ce-4370-b6d9-eb248cb8af8f","Type":"ContainerDied","Data":"6583335451a309a8fd64d0f9ae1ce475846669cd1b3a8ddc61fee52ade95897d"}
Sep 18 18:26:18.122828 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:18.122779    2568 scope.go:117] "RemoveContainer" containerID="1b33f07f4a0f76acb47f16eb83929cd61dad3c6c4db354cb563d02ea6fba328b"
Sep 18 18:26:18.131986 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:18.131968    2568 scope.go:117] "RemoveContainer" containerID="2b818ac88151ba56bcb8a04f67de0a28d5887507426ae4602b36ee8e08edaa74"
Sep 18 18:26:18.140156 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:18.140133    2568 scope.go:117] "RemoveContainer" containerID="1b33f07f4a0f76acb47f16eb83929cd61dad3c6c4db354cb563d02ea6fba328b"
Sep 18 18:26:18.140479 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:26:18.140461    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b33f07f4a0f76acb47f16eb83929cd61dad3c6c4db354cb563d02ea6fba328b\": container with ID starting with 1b33f07f4a0f76acb47f16eb83929cd61dad3c6c4db354cb563d02ea6fba328b not found: ID does not exist" containerID="1b33f07f4a0f76acb47f16eb83929cd61dad3c6c4db354cb563d02ea6fba328b"
Sep 18 18:26:18.140557 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:18.140493    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b33f07f4a0f76acb47f16eb83929cd61dad3c6c4db354cb563d02ea6fba328b"} err="failed to get container status \"1b33f07f4a0f76acb47f16eb83929cd61dad3c6c4db354cb563d02ea6fba328b\": rpc error: code = NotFound desc = could not find container \"1b33f07f4a0f76acb47f16eb83929cd61dad3c6c4db354cb563d02ea6fba328b\": container with ID starting with 1b33f07f4a0f76acb47f16eb83929cd61dad3c6c4db354cb563d02ea6fba328b not found: ID does not exist"
Sep 18 18:26:18.140557 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:18.140516    2568 scope.go:117] "RemoveContainer" containerID="2b818ac88151ba56bcb8a04f67de0a28d5887507426ae4602b36ee8e08edaa74"
Sep 18 18:26:18.140803 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:26:18.140779    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b818ac88151ba56bcb8a04f67de0a28d5887507426ae4602b36ee8e08edaa74\": container with ID starting with 2b818ac88151ba56bcb8a04f67de0a28d5887507426ae4602b36ee8e08edaa74 not found: ID does not exist" containerID="2b818ac88151ba56bcb8a04f67de0a28d5887507426ae4602b36ee8e08edaa74"
Sep 18 18:26:18.140848 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:18.140813    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b818ac88151ba56bcb8a04f67de0a28d5887507426ae4602b36ee8e08edaa74"} err="failed to get container status \"2b818ac88151ba56bcb8a04f67de0a28d5887507426ae4602b36ee8e08edaa74\": rpc error: code = NotFound desc = could not find container \"2b818ac88151ba56bcb8a04f67de0a28d5887507426ae4602b36ee8e08edaa74\": container with ID starting with 2b818ac88151ba56bcb8a04f67de0a28d5887507426ae4602b36ee8e08edaa74 not found: ID does not exist"
Sep 18 18:26:18.141160 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:18.141145    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"]
Sep 18 18:26:18.144214 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:18.144191    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-53cde3-openshift-default-8666779df4-gtdrm"]
Sep 18 18:26:20.045451 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:20.045418    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e083287-05ce-4370-b6d9-eb248cb8af8f" path="/var/lib/kubelet/pods/4e083287-05ce-4370-b6d9-eb248cb8af8f/volumes"
Sep 18 18:26:22.832499 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:22.832464    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"]
Sep 18 18:26:22.833381 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:22.833344    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb" podUID="99dda42d-cc46-4c2f-b4a4-08d979dfd7b5" containerName="maas-api" containerID="cri-o://66ca0e49873d29d213b7f5983ecd4fa1c37076775ee038bfa1a5923a67e46ed8" gracePeriod=30
Sep 18 18:26:22.923297 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:22.923253    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb" podUID="99dda42d-cc46-4c2f-b4a4-08d979dfd7b5" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.72:8443/health\": dial tcp 10.134.0.72:8443: connect: connection refused"
Sep 18 18:26:23.090536 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.090479    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:26:23.154403 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.154363    2568 generic.go:356] "Generic (PLEG): container finished" podID="99dda42d-cc46-4c2f-b4a4-08d979dfd7b5" containerID="66ca0e49873d29d213b7f5983ecd4fa1c37076775ee038bfa1a5923a67e46ed8" exitCode=0
Sep 18 18:26:23.154594 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.154406    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"
Sep 18 18:26:23.154594 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.154424    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb" event={"ID":"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5","Type":"ContainerDied","Data":"66ca0e49873d29d213b7f5983ecd4fa1c37076775ee038bfa1a5923a67e46ed8"}
Sep 18 18:26:23.154594 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.154471    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb" event={"ID":"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5","Type":"ContainerDied","Data":"b2f657aa88a0b9ba5f28fbcaac8f453edc7467a52b2120d10ed40602a0022f46"}
Sep 18 18:26:23.154594 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.154497    2568 scope.go:117] "RemoveContainer" containerID="66ca0e49873d29d213b7f5983ecd4fa1c37076775ee038bfa1a5923a67e46ed8"
Sep 18 18:26:23.158755 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.158729    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmc7v\" (UniqueName: \"kubernetes.io/projected/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-kube-api-access-xmc7v\") pod \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") "
Sep 18 18:26:23.158897 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.158815    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-openshift-service-ca\") pod \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") "
Sep 18 18:26:23.158897 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.158868    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-metrics-certs\") pod \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") "
Sep 18 18:26:23.159026 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.158928    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-maas-api-tls\") pod \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\" (UID: \"99dda42d-cc46-4c2f-b4a4-08d979dfd7b5\") "
Sep 18 18:26:23.159835 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.159787    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "99dda42d-cc46-4c2f-b4a4-08d979dfd7b5" (UID: "99dda42d-cc46-4c2f-b4a4-08d979dfd7b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:26:23.161604 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.161577    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-kube-api-access-xmc7v" (OuterVolumeSpecName: "kube-api-access-xmc7v") pod "99dda42d-cc46-4c2f-b4a4-08d979dfd7b5" (UID: "99dda42d-cc46-4c2f-b4a4-08d979dfd7b5"). InnerVolumeSpecName "kube-api-access-xmc7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:23.161830 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.161797    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "99dda42d-cc46-4c2f-b4a4-08d979dfd7b5" (UID: "99dda42d-cc46-4c2f-b4a4-08d979dfd7b5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:26:23.161913 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.161883    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "99dda42d-cc46-4c2f-b4a4-08d979dfd7b5" (UID: "99dda42d-cc46-4c2f-b4a4-08d979dfd7b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:26:23.166074 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.166054    2568 scope.go:117] "RemoveContainer" containerID="66ca0e49873d29d213b7f5983ecd4fa1c37076775ee038bfa1a5923a67e46ed8"
Sep 18 18:26:23.166380 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:26:23.166360    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66ca0e49873d29d213b7f5983ecd4fa1c37076775ee038bfa1a5923a67e46ed8\": container with ID starting with 66ca0e49873d29d213b7f5983ecd4fa1c37076775ee038bfa1a5923a67e46ed8 not found: ID does not exist" containerID="66ca0e49873d29d213b7f5983ecd4fa1c37076775ee038bfa1a5923a67e46ed8"
Sep 18 18:26:23.166446 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.166387    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66ca0e49873d29d213b7f5983ecd4fa1c37076775ee038bfa1a5923a67e46ed8"} err="failed to get container status \"66ca0e49873d29d213b7f5983ecd4fa1c37076775ee038bfa1a5923a67e46ed8\": rpc error: code = NotFound desc = could not find container \"66ca0e49873d29d213b7f5983ecd4fa1c37076775ee038bfa1a5923a67e46ed8\": container with ID starting with 66ca0e49873d29d213b7f5983ecd4fa1c37076775ee038bfa1a5923a67e46ed8 not found: ID does not exist"
Sep 18 18:26:23.260491 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.260460    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:23.260491 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.260484    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmc7v\" (UniqueName: \"kubernetes.io/projected/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-kube-api-access-xmc7v\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:23.260491 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.260495    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:23.260692 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.260504    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5-metrics-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:23.477273 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.477244    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"]
Sep 18 18:26:23.479251 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:23.479232    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-341bd4-5fd9595d4d-9fzqb"]
Sep 18 18:26:24.045004 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:24.044972    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99dda42d-cc46-4c2f-b4a4-08d979dfd7b5" path="/var/lib/kubelet/pods/99dda42d-cc46-4c2f-b4a4-08d979dfd7b5/volumes"
Sep 18 18:26:25.714590 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:25.714559    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:26:29.443526 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:29.443496    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"]
Sep 18 18:26:29.443931 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:29.443689    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq" podUID="b5c0e9bd-e389-4c85-9f50-75c3cae1c218" containerName="istio-proxy" containerID="cri-o://46537d90c0b5c8224a0941b3ebde9b2212a5af9561cbb22193dc5ddec7cce77c" gracePeriod=30
Sep 18 18:26:34.699806 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.699781    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:26:34.762434 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.762369    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-podinfo\") pod \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") "
Sep 18 18:26:34.762551 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.762432    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-workload-certs\") pod \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") "
Sep 18 18:26:34.762551 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.762463    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-token\") pod \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") "
Sep 18 18:26:34.762551 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.762511    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istiod-ca-cert\") pod \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") "
Sep 18 18:26:34.762551 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.762539    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4c2r9\" (UniqueName: \"kubernetes.io/projected/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-kube-api-access-4c2r9\") pod \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") "
Sep 18 18:26:34.762702 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.762566    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-credential-socket\") pod \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") "
Sep 18 18:26:34.762702 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.762615    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-envoy\") pod \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") "
Sep 18 18:26:34.762702 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.762652    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-workload-socket\") pod \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") "
Sep 18 18:26:34.762702 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.762686    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-data\") pod \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\" (UID: \"b5c0e9bd-e389-4c85-9f50-75c3cae1c218\") "
Sep 18 18:26:34.762897 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.762757    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b5c0e9bd-e389-4c85-9f50-75c3cae1c218" (UID: "b5c0e9bd-e389-4c85-9f50-75c3cae1c218"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:34.762897 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.762862    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b5c0e9bd-e389-4c85-9f50-75c3cae1c218" (UID: "b5c0e9bd-e389-4c85-9f50-75c3cae1c218"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:26:34.763005 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.762965    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b5c0e9bd-e389-4c85-9f50-75c3cae1c218" (UID: "b5c0e9bd-e389-4c85-9f50-75c3cae1c218"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:34.763250 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.763143    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-workload-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:34.763250 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.763169    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istiod-ca-cert\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:34.763250 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.763217    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-credential-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:34.763250 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.763165    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b5c0e9bd-e389-4c85-9f50-75c3cae1c218" (UID: "b5c0e9bd-e389-4c85-9f50-75c3cae1c218"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:34.763250 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.763237    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-data" (OuterVolumeSpecName: "istio-data") pod "b5c0e9bd-e389-4c85-9f50-75c3cae1c218" (UID: "b5c0e9bd-e389-4c85-9f50-75c3cae1c218"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:34.764684 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.764658    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b5c0e9bd-e389-4c85-9f50-75c3cae1c218" (UID: "b5c0e9bd-e389-4c85-9f50-75c3cae1c218"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:26:34.764888 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.764874    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b5c0e9bd-e389-4c85-9f50-75c3cae1c218" (UID: "b5c0e9bd-e389-4c85-9f50-75c3cae1c218"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:34.764888 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.764873    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-kube-api-access-4c2r9" (OuterVolumeSpecName: "kube-api-access-4c2r9") pod "b5c0e9bd-e389-4c85-9f50-75c3cae1c218" (UID: "b5c0e9bd-e389-4c85-9f50-75c3cae1c218"). InnerVolumeSpecName "kube-api-access-4c2r9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:34.764989 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.764972    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-token" (OuterVolumeSpecName: "istio-token") pod "b5c0e9bd-e389-4c85-9f50-75c3cae1c218" (UID: "b5c0e9bd-e389-4c85-9f50-75c3cae1c218"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:34.864605 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.864571    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-podinfo\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:34.864605 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.864599    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-token\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:34.864605 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.864608    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4c2r9\" (UniqueName: \"kubernetes.io/projected/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-kube-api-access-4c2r9\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:34.864824 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.864620    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-envoy\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:34.864824 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.864629    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-workload-socket\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:34.864824 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:34.864636    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5c0e9bd-e389-4c85-9f50-75c3cae1c218-istio-data\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:26:35.222834 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:35.222797    2568 generic.go:356] "Generic (PLEG): container finished" podID="b5c0e9bd-e389-4c85-9f50-75c3cae1c218" containerID="46537d90c0b5c8224a0941b3ebde9b2212a5af9561cbb22193dc5ddec7cce77c" exitCode=0
Sep 18 18:26:35.223023 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:35.222854    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"
Sep 18 18:26:35.223023 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:35.222885    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq" event={"ID":"b5c0e9bd-e389-4c85-9f50-75c3cae1c218","Type":"ContainerDied","Data":"46537d90c0b5c8224a0941b3ebde9b2212a5af9561cbb22193dc5ddec7cce77c"}
Sep 18 18:26:35.223023 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:35.222943    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq" event={"ID":"b5c0e9bd-e389-4c85-9f50-75c3cae1c218","Type":"ContainerDied","Data":"336ddb3af1b8738bc4ea29ed6a039e8086fdcaae9edeab4411b6abb42ffbe681"}
Sep 18 18:26:35.223023 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:35.222964    2568 scope.go:117] "RemoveContainer" containerID="46537d90c0b5c8224a0941b3ebde9b2212a5af9561cbb22193dc5ddec7cce77c"
Sep 18 18:26:35.232193 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:35.232162    2568 scope.go:117] "RemoveContainer" containerID="f002ba0910ba0f02e461fc12d01ccf1bff92581a4dcfa7bf6a4011641ec14488"
Sep 18 18:26:35.240810 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:35.240783    2568 scope.go:117] "RemoveContainer" containerID="46537d90c0b5c8224a0941b3ebde9b2212a5af9561cbb22193dc5ddec7cce77c"
Sep 18 18:26:35.241100 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:26:35.241081    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46537d90c0b5c8224a0941b3ebde9b2212a5af9561cbb22193dc5ddec7cce77c\": container with ID starting with 46537d90c0b5c8224a0941b3ebde9b2212a5af9561cbb22193dc5ddec7cce77c not found: ID does not exist" containerID="46537d90c0b5c8224a0941b3ebde9b2212a5af9561cbb22193dc5ddec7cce77c"
Sep 18 18:26:35.241156 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:35.241109    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46537d90c0b5c8224a0941b3ebde9b2212a5af9561cbb22193dc5ddec7cce77c"} err="failed to get container status \"46537d90c0b5c8224a0941b3ebde9b2212a5af9561cbb22193dc5ddec7cce77c\": rpc error: code = NotFound desc = could not find container \"46537d90c0b5c8224a0941b3ebde9b2212a5af9561cbb22193dc5ddec7cce77c\": container with ID starting with 46537d90c0b5c8224a0941b3ebde9b2212a5af9561cbb22193dc5ddec7cce77c not found: ID does not exist"
Sep 18 18:26:35.241156 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:35.241128    2568 scope.go:117] "RemoveContainer" containerID="f002ba0910ba0f02e461fc12d01ccf1bff92581a4dcfa7bf6a4011641ec14488"
Sep 18 18:26:35.241355 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:26:35.241341    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f002ba0910ba0f02e461fc12d01ccf1bff92581a4dcfa7bf6a4011641ec14488\": container with ID starting with f002ba0910ba0f02e461fc12d01ccf1bff92581a4dcfa7bf6a4011641ec14488 not found: ID does not exist" containerID="f002ba0910ba0f02e461fc12d01ccf1bff92581a4dcfa7bf6a4011641ec14488"
Sep 18 18:26:35.241394 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:35.241358    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f002ba0910ba0f02e461fc12d01ccf1bff92581a4dcfa7bf6a4011641ec14488"} err="failed to get container status \"f002ba0910ba0f02e461fc12d01ccf1bff92581a4dcfa7bf6a4011641ec14488\": rpc error: code = NotFound desc = could not find container \"f002ba0910ba0f02e461fc12d01ccf1bff92581a4dcfa7bf6a4011641ec14488\": container with ID starting with f002ba0910ba0f02e461fc12d01ccf1bff92581a4dcfa7bf6a4011641ec14488 not found: ID does not exist"
Sep 18 18:26:35.247200 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:35.247166    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"]
Sep 18 18:26:35.250799 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:35.250780    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-341bd4-openshift-default-7947d6ccd6-gq9jq"]
Sep 18 18:26:35.324169 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:35.324140    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:26:36.044520 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:36.044489    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5c0e9bd-e389-4c85-9f50-75c3cae1c218" path="/var/lib/kubelet/pods/b5c0e9bd-e389-4c85-9f50-75c3cae1c218/volumes"
Sep 18 18:26:58.404155 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:26:58.404120    2568 scope.go:117] "RemoveContainer" containerID="f6ccd687aaaf04ae5eb4a83e720b5d9d540e911cb98eff2edf94a1a9c430d4dc"
Sep 18 18:27:56.013881 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:27:56.013847    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:28:40.016013 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:28:40.015971    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:28:47.021946 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:28:47.021862    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:28:53.116939 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:28:53.116899    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:29:00.227470 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:00.227427    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:29:08.610784 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:08.610741    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:29:22.820680 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:22.820644    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:29:28.515609 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:28.515577    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:29:30.268941 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:30.268911    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"]
Sep 18 18:29:30.269433 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:30.269111    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2" podUID="dc902a56-b126-4d91-9eab-14c0eb02358a" containerName="manager" containerID="cri-o://e1b23fb41434bf77dccac2f7c9d68bb44968ce6ec6775556d348f3da3ff059e4" gracePeriod=10
Sep 18 18:29:31.179540 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:31.179509    2568 generic.go:356] "Generic (PLEG): container finished" podID="dc902a56-b126-4d91-9eab-14c0eb02358a" containerID="e1b23fb41434bf77dccac2f7c9d68bb44968ce6ec6775556d348f3da3ff059e4" exitCode=0
Sep 18 18:29:31.179695 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:31.179584    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2" event={"ID":"dc902a56-b126-4d91-9eab-14c0eb02358a","Type":"ContainerDied","Data":"e1b23fb41434bf77dccac2f7c9d68bb44968ce6ec6775556d348f3da3ff059e4"}
Sep 18 18:29:31.208648 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:31.208623    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"
Sep 18 18:29:31.287973 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:31.287946    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x47gh\" (UniqueName: \"kubernetes.io/projected/dc902a56-b126-4d91-9eab-14c0eb02358a-kube-api-access-x47gh\") pod \"dc902a56-b126-4d91-9eab-14c0eb02358a\" (UID: \"dc902a56-b126-4d91-9eab-14c0eb02358a\") "
Sep 18 18:29:31.287973 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:31.287979    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc902a56-b126-4d91-9eab-14c0eb02358a-extensions-socket-volume\") pod \"dc902a56-b126-4d91-9eab-14c0eb02358a\" (UID: \"dc902a56-b126-4d91-9eab-14c0eb02358a\") "
Sep 18 18:29:31.288486 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:31.288368    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc902a56-b126-4d91-9eab-14c0eb02358a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dc902a56-b126-4d91-9eab-14c0eb02358a" (UID: "dc902a56-b126-4d91-9eab-14c0eb02358a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:29:31.290056 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:31.290033    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc902a56-b126-4d91-9eab-14c0eb02358a-kube-api-access-x47gh" (OuterVolumeSpecName: "kube-api-access-x47gh") pod "dc902a56-b126-4d91-9eab-14c0eb02358a" (UID: "dc902a56-b126-4d91-9eab-14c0eb02358a"). InnerVolumeSpecName "kube-api-access-x47gh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:29:31.389591 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:31.389525    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x47gh\" (UniqueName: \"kubernetes.io/projected/dc902a56-b126-4d91-9eab-14c0eb02358a-kube-api-access-x47gh\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:29:31.389591 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:31.389551    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc902a56-b126-4d91-9eab-14c0eb02358a-extensions-socket-volume\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:29:32.186454 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:32.186423    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"
Sep 18 18:29:32.186454 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:32.186443    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2" event={"ID":"dc902a56-b126-4d91-9eab-14c0eb02358a","Type":"ContainerDied","Data":"166594baa274bfd59335ef0021c82df19b1ac07028cd0bbcfb2a55defdb655d1"}
Sep 18 18:29:32.186665 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:32.186498    2568 scope.go:117] "RemoveContainer" containerID="e1b23fb41434bf77dccac2f7c9d68bb44968ce6ec6775556d348f3da3ff059e4"
Sep 18 18:29:32.209678 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:32.205995    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"]
Sep 18 18:29:32.214248 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:32.214225    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2qqd2"]
Sep 18 18:29:34.044761 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:34.044730    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc902a56-b126-4d91-9eab-14c0eb02358a" path="/var/lib/kubelet/pods/dc902a56-b126-4d91-9eab-14c0eb02358a/volumes"
Sep 18 18:29:58.192405 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:58.192378    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 18:29:58.203225 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:29:58.203202    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-56mpn_bffad327-4f7a-40a1-b969-01a7799008f1/ovn-acl-logging/0.log"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.142392    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829270-nfgcp"]
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143263    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb151f4-9d47-4427-bc53-c7d5f0d6c84f" containerName="main"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143283    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb151f4-9d47-4427-bc53-c7d5f0d6c84f" containerName="main"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143295    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143305    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143330    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143339    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143363    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143372    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143386    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5c0e9bd-e389-4c85-9f50-75c3cae1c218" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143397    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5c0e9bd-e389-4c85-9f50-75c3cae1c218" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143416    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc902a56-b126-4d91-9eab-14c0eb02358a" containerName="manager"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143424    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc902a56-b126-4d91-9eab-14c0eb02358a" containerName="manager"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143453    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99dda42d-cc46-4c2f-b4a4-08d979dfd7b5" containerName="maas-api"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143461    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="99dda42d-cc46-4c2f-b4a4-08d979dfd7b5" containerName="maas-api"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143469    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d66e3772-dd2d-4d90-8014-3aca2be939bc" containerName="maas-api"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143478    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d66e3772-dd2d-4d90-8014-3aca2be939bc" containerName="maas-api"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143497    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5c0e9bd-e389-4c85-9f50-75c3cae1c218" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143505    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5c0e9bd-e389-4c85-9f50-75c3cae1c218" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143523    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143531    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143736    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb151f4-9d47-4427-bc53-c7d5f0d6c84f" containerName="main"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143748    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d66e3772-dd2d-4d90-8014-3aca2be939bc" containerName="maas-api"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143758    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143773    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="99dda42d-cc46-4c2f-b4a4-08d979dfd7b5" containerName="maas-api"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143790    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc902a56-b126-4d91-9eab-14c0eb02358a" containerName="manager"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143799    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5c0e9bd-e389-4c85-9f50-75c3cae1c218" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143807    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143814    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e083287-05ce-4370-b6d9-eb248cb8af8f" containerName="istio-proxy"
Sep 18 18:30:00.145152 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.143829    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5c0e9bd-e389-4c85-9f50-75c3cae1c218" containerName="istio-proxy"
Sep 18 18:30:00.148094 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.148063    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829270-nfgcp"
Sep 18 18:30:00.148677 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.148639    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829270-nfgcp"]
Sep 18 18:30:00.150506 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.150482    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-88z7v\""
Sep 18 18:30:00.158580 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.158563    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829270-nfgcp"
Sep 18 18:30:00.427537 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:00.427314    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829270-nfgcp"]
Sep 18 18:30:00.429809 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:30:00.429783    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc074e408_98ce_4aa9_800a_f623990faea1.slice/crio-c3d3bb12632a52edf4b0a33769011069867eff6715b74d6c4b78179beeef2a3d WatchSource:0}: Error finding container c3d3bb12632a52edf4b0a33769011069867eff6715b74d6c4b78179beeef2a3d: Status 404 returned error can't find the container with id c3d3bb12632a52edf4b0a33769011069867eff6715b74d6c4b78179beeef2a3d
Sep 18 18:30:01.333487 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:01.333455    2568 generic.go:356] "Generic (PLEG): container finished" podID="c074e408-98ce-4aa9-800a-f623990faea1" containerID="f137eca3ef251fb602f182f5951fb83caaee0cecbc5353ce78d4550e2190ba63" exitCode=0
Sep 18 18:30:01.333897 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:01.333517    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829270-nfgcp" event={"ID":"c074e408-98ce-4aa9-800a-f623990faea1","Type":"ContainerDied","Data":"f137eca3ef251fb602f182f5951fb83caaee0cecbc5353ce78d4550e2190ba63"}
Sep 18 18:30:01.333897 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:01.333553    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829270-nfgcp" event={"ID":"c074e408-98ce-4aa9-800a-f623990faea1","Type":"ContainerStarted","Data":"c3d3bb12632a52edf4b0a33769011069867eff6715b74d6c4b78179beeef2a3d"}
Sep 18 18:30:02.473073 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:02.473043    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829270-nfgcp"
Sep 18 18:30:03.344388 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:03.344356    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829270-nfgcp"
Sep 18 18:30:03.344388 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:03.344362    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829270-nfgcp" event={"ID":"c074e408-98ce-4aa9-800a-f623990faea1","Type":"ContainerDied","Data":"c3d3bb12632a52edf4b0a33769011069867eff6715b74d6c4b78179beeef2a3d"}
Sep 18 18:30:03.344617 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:03.344414    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3d3bb12632a52edf4b0a33769011069867eff6715b74d6c4b78179beeef2a3d"
Sep 18 18:30:40.943716 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:40.943679    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb"]
Sep 18 18:30:40.944228 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:40.944081    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c074e408-98ce-4aa9-800a-f623990faea1" containerName="cleanup"
Sep 18 18:30:40.944228 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:40.944091    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c074e408-98ce-4aa9-800a-f623990faea1" containerName="cleanup"
Sep 18 18:30:40.944228 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:40.944147    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c074e408-98ce-4aa9-800a-f623990faea1" containerName="cleanup"
Sep 18 18:30:40.944228 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:40.944156    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="54e6baeb-e39c-45dd-9b49-6cf09eff3dbe" containerName="istio-proxy"
Sep 18 18:30:40.950637 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:40.949459    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb"
Sep 18 18:30:40.953650 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:40.953621    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9x8zw\""
Sep 18 18:30:40.959130 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:40.959102    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb"]
Sep 18 18:30:41.125463 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:41.125423    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d412c9ec-3863-4e69-8d05-db156736d3d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fq2mb\" (UID: \"d412c9ec-3863-4e69-8d05-db156736d3d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb"
Sep 18 18:30:41.125642 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:41.125480    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c24r9\" (UniqueName: \"kubernetes.io/projected/d412c9ec-3863-4e69-8d05-db156736d3d8-kube-api-access-c24r9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fq2mb\" (UID: \"d412c9ec-3863-4e69-8d05-db156736d3d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb"
Sep 18 18:30:41.226822 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:41.226713    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d412c9ec-3863-4e69-8d05-db156736d3d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fq2mb\" (UID: \"d412c9ec-3863-4e69-8d05-db156736d3d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb"
Sep 18 18:30:41.226822 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:41.226810    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c24r9\" (UniqueName: \"kubernetes.io/projected/d412c9ec-3863-4e69-8d05-db156736d3d8-kube-api-access-c24r9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fq2mb\" (UID: \"d412c9ec-3863-4e69-8d05-db156736d3d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb"
Sep 18 18:30:41.227231 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:41.227171    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d412c9ec-3863-4e69-8d05-db156736d3d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fq2mb\" (UID: \"d412c9ec-3863-4e69-8d05-db156736d3d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb"
Sep 18 18:30:41.238093 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:41.238067    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c24r9\" (UniqueName: \"kubernetes.io/projected/d412c9ec-3863-4e69-8d05-db156736d3d8-kube-api-access-c24r9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fq2mb\" (UID: \"d412c9ec-3863-4e69-8d05-db156736d3d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb"
Sep 18 18:30:41.261665 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:41.261639    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb"
Sep 18 18:30:41.541247 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:41.539004    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb"]
Sep 18 18:30:41.543053 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:30:41.543023    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd412c9ec_3863_4e69_8d05_db156736d3d8.slice/crio-7f50f2efdd862e84973aaaa47dad5fe04689968a31da8c940c06a72ae7e73464 WatchSource:0}: Error finding container 7f50f2efdd862e84973aaaa47dad5fe04689968a31da8c940c06a72ae7e73464: Status 404 returned error can't find the container with id 7f50f2efdd862e84973aaaa47dad5fe04689968a31da8c940c06a72ae7e73464
Sep 18 18:30:42.548756 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:42.548714    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb" event={"ID":"d412c9ec-3863-4e69-8d05-db156736d3d8","Type":"ContainerStarted","Data":"647d90d8da37c7f72bbc2432fd5762dff321a6d4618b0a9fb1d790a508c4000e"}
Sep 18 18:30:42.549130 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:42.548767    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb" event={"ID":"d412c9ec-3863-4e69-8d05-db156736d3d8","Type":"ContainerStarted","Data":"7f50f2efdd862e84973aaaa47dad5fe04689968a31da8c940c06a72ae7e73464"}
Sep 18 18:30:42.549130 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:42.548851    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb"
Sep 18 18:30:42.578871 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:42.578818    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb" podStartSLOduration=2.5787999729999997 podStartE2EDuration="2.578799973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:30:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:30:42.57639937 +0000 UTC m=+2446.654203269" watchObservedRunningTime="2026-09-18 18:30:42.578799973 +0000 UTC m=+2446.656603872"
Sep 18 18:30:53.557008 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:30:53.556975    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fq2mb"
Sep 18 18:31:05.309638 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:31:05.309605    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:31:11.119051 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:31:11.119012    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:31:18.720207 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:31:18.718876    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:31:30.616142 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:31:30.616098    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:31:50.723452 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:31:50.723414    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:31:59.607580 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:31:59.607544    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:32:08.919136 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:08.919096    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:32:09.306052 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.305968    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"]
Sep 18 18:32:09.309638 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.309622    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:09.312791 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.312769    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 18:32:09.312791 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.312775    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:32:09.312791 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.312785    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 18 18:32:09.313043 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.312811    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tlsbx\""
Sep 18 18:32:09.313043 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.312855    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 18:32:09.328277 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.328253    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"]
Sep 18 18:32:09.340978 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.340956    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwcd7\" (UniqueName: \"kubernetes.io/projected/ad25a6b3-6570-43d7-b14a-e035ceb1808b-kube-api-access-qwcd7\") pod \"maas-api-7dddcbdd5b-zgbjc\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") " pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:09.341109 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.341013    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad25a6b3-6570-43d7-b14a-e035ceb1808b-metrics-certs\") pod \"maas-api-7dddcbdd5b-zgbjc\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") " pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:09.341156 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.341118    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad25a6b3-6570-43d7-b14a-e035ceb1808b-maas-api-tls\") pod \"maas-api-7dddcbdd5b-zgbjc\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") " pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:09.341229 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.341200    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad25a6b3-6570-43d7-b14a-e035ceb1808b-openshift-service-ca\") pod \"maas-api-7dddcbdd5b-zgbjc\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") " pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:09.441951 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.441900    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad25a6b3-6570-43d7-b14a-e035ceb1808b-metrics-certs\") pod \"maas-api-7dddcbdd5b-zgbjc\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") " pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:09.442158 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.442069    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad25a6b3-6570-43d7-b14a-e035ceb1808b-maas-api-tls\") pod \"maas-api-7dddcbdd5b-zgbjc\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") " pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:09.442158 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.442132    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad25a6b3-6570-43d7-b14a-e035ceb1808b-openshift-service-ca\") pod \"maas-api-7dddcbdd5b-zgbjc\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") " pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:09.442323 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.442231    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwcd7\" (UniqueName: \"kubernetes.io/projected/ad25a6b3-6570-43d7-b14a-e035ceb1808b-kube-api-access-qwcd7\") pod \"maas-api-7dddcbdd5b-zgbjc\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") " pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:09.442875 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.442851    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad25a6b3-6570-43d7-b14a-e035ceb1808b-openshift-service-ca\") pod \"maas-api-7dddcbdd5b-zgbjc\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") " pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:09.444471 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.444453    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad25a6b3-6570-43d7-b14a-e035ceb1808b-metrics-certs\") pod \"maas-api-7dddcbdd5b-zgbjc\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") " pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:09.444551 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.444501    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad25a6b3-6570-43d7-b14a-e035ceb1808b-maas-api-tls\") pod \"maas-api-7dddcbdd5b-zgbjc\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") " pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:09.449364 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.449348    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwcd7\" (UniqueName: \"kubernetes.io/projected/ad25a6b3-6570-43d7-b14a-e035ceb1808b-kube-api-access-qwcd7\") pod \"maas-api-7dddcbdd5b-zgbjc\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") " pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:09.620446 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.620349    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:09.894769 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.894743    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"]
Sep 18 18:32:09.896533 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:32:09.896508    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad25a6b3_6570_43d7_b14a_e035ceb1808b.slice/crio-b33726373e02967e95ce6eb4681f6a4d8ad05c5a7dc60f0c80057508042291db WatchSource:0}: Error finding container b33726373e02967e95ce6eb4681f6a4d8ad05c5a7dc60f0c80057508042291db: Status 404 returned error can't find the container with id b33726373e02967e95ce6eb4681f6a4d8ad05c5a7dc60f0c80057508042291db
Sep 18 18:32:09.897835 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:09.897813    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:32:10.020112 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:10.020079    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc" event={"ID":"ad25a6b3-6570-43d7-b14a-e035ceb1808b","Type":"ContainerStarted","Data":"b33726373e02967e95ce6eb4681f6a4d8ad05c5a7dc60f0c80057508042291db"}
Sep 18 18:32:11.026986 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:11.026950    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc" event={"ID":"ad25a6b3-6570-43d7-b14a-e035ceb1808b","Type":"ContainerStarted","Data":"4c7e111730f9a73985a06a3d498c946bc1596bcd9ff47dc7102cc9e01e9d1631"}
Sep 18 18:32:11.027406 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:11.027091    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:11.046955 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:11.046897    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc" podStartSLOduration=1.5532610390000001 podStartE2EDuration="2.046879179s" totalImagesPullingTime="493.61814ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:32:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:32:11.045012995 +0000 UTC m=+2535.122816895" watchObservedRunningTime="2026-09-18 18:32:11.046879179 +0000 UTC m=+2535.124683082"
Sep 18 18:32:16.617776 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:16.617739    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:32:16.763583 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:16.763543    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"]
Sep 18 18:32:16.763844 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:16.763814    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc" podUID="ad25a6b3-6570-43d7-b14a-e035ceb1808b" containerName="maas-api" containerID="cri-o://4c7e111730f9a73985a06a3d498c946bc1596bcd9ff47dc7102cc9e01e9d1631" gracePeriod=30
Sep 18 18:32:16.769668 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:16.769647    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:16.797694 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:16.794537    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"]
Sep 18 18:32:16.801307 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:16.800109    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:16.808168 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:16.808057    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"]
Sep 18 18:32:16.910596 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:16.910563    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6-openshift-service-ca\") pod \"maas-api-58fcc54d4-xb28t\" (UID: \"bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6\") " pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:16.910774 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:16.910672    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf7hr\" (UniqueName: \"kubernetes.io/projected/bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6-kube-api-access-xf7hr\") pod \"maas-api-58fcc54d4-xb28t\" (UID: \"bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6\") " pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:16.910845 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:16.910761    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6-maas-api-tls\") pod \"maas-api-58fcc54d4-xb28t\" (UID: \"bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6\") " pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:16.910845 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:16.910797    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6-metrics-certs\") pod \"maas-api-58fcc54d4-xb28t\" (UID: \"bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6\") " pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:17.008124 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.008095    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:17.011885 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.011859    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf7hr\" (UniqueName: \"kubernetes.io/projected/bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6-kube-api-access-xf7hr\") pod \"maas-api-58fcc54d4-xb28t\" (UID: \"bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6\") " pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:17.012004 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.011931    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6-maas-api-tls\") pod \"maas-api-58fcc54d4-xb28t\" (UID: \"bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6\") " pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:17.012004 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.011955    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6-metrics-certs\") pod \"maas-api-58fcc54d4-xb28t\" (UID: \"bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6\") " pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:17.012004 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.011979    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6-openshift-service-ca\") pod \"maas-api-58fcc54d4-xb28t\" (UID: \"bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6\") " pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:17.012610 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.012589    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6-openshift-service-ca\") pod \"maas-api-58fcc54d4-xb28t\" (UID: \"bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6\") " pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:17.014294 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.014274    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6-metrics-certs\") pod \"maas-api-58fcc54d4-xb28t\" (UID: \"bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6\") " pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:17.014480 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.014460    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6-maas-api-tls\") pod \"maas-api-58fcc54d4-xb28t\" (UID: \"bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6\") " pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:17.019884 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.019861    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf7hr\" (UniqueName: \"kubernetes.io/projected/bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6-kube-api-access-xf7hr\") pod \"maas-api-58fcc54d4-xb28t\" (UID: \"bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6\") " pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:17.060096 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.060064    2568 generic.go:356] "Generic (PLEG): container finished" podID="ad25a6b3-6570-43d7-b14a-e035ceb1808b" containerID="4c7e111730f9a73985a06a3d498c946bc1596bcd9ff47dc7102cc9e01e9d1631" exitCode=0
Sep 18 18:32:17.060264 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.060120    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"
Sep 18 18:32:17.060264 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.060143    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc" event={"ID":"ad25a6b3-6570-43d7-b14a-e035ceb1808b","Type":"ContainerDied","Data":"4c7e111730f9a73985a06a3d498c946bc1596bcd9ff47dc7102cc9e01e9d1631"}
Sep 18 18:32:17.060264 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.060203    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc" event={"ID":"ad25a6b3-6570-43d7-b14a-e035ceb1808b","Type":"ContainerDied","Data":"b33726373e02967e95ce6eb4681f6a4d8ad05c5a7dc60f0c80057508042291db"}
Sep 18 18:32:17.060264 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.060231    2568 scope.go:117] "RemoveContainer" containerID="4c7e111730f9a73985a06a3d498c946bc1596bcd9ff47dc7102cc9e01e9d1631"
Sep 18 18:32:17.068842 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.068827    2568 scope.go:117] "RemoveContainer" containerID="4c7e111730f9a73985a06a3d498c946bc1596bcd9ff47dc7102cc9e01e9d1631"
Sep 18 18:32:17.069111 ip-10-0-130-66 kubenswrapper[2568]: E0918 18:32:17.069092    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c7e111730f9a73985a06a3d498c946bc1596bcd9ff47dc7102cc9e01e9d1631\": container with ID starting with 4c7e111730f9a73985a06a3d498c946bc1596bcd9ff47dc7102cc9e01e9d1631 not found: ID does not exist" containerID="4c7e111730f9a73985a06a3d498c946bc1596bcd9ff47dc7102cc9e01e9d1631"
Sep 18 18:32:17.069208 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.069123    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c7e111730f9a73985a06a3d498c946bc1596bcd9ff47dc7102cc9e01e9d1631"} err="failed to get container status \"4c7e111730f9a73985a06a3d498c946bc1596bcd9ff47dc7102cc9e01e9d1631\": rpc error: code = NotFound desc = could not find container \"4c7e111730f9a73985a06a3d498c946bc1596bcd9ff47dc7102cc9e01e9d1631\": container with ID starting with 4c7e111730f9a73985a06a3d498c946bc1596bcd9ff47dc7102cc9e01e9d1631 not found: ID does not exist"
Sep 18 18:32:17.112961 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.112935    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwcd7\" (UniqueName: \"kubernetes.io/projected/ad25a6b3-6570-43d7-b14a-e035ceb1808b-kube-api-access-qwcd7\") pod \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") "
Sep 18 18:32:17.113039 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.113015    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad25a6b3-6570-43d7-b14a-e035ceb1808b-openshift-service-ca\") pod \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") "
Sep 18 18:32:17.113087 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.113062    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad25a6b3-6570-43d7-b14a-e035ceb1808b-metrics-certs\") pod \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") "
Sep 18 18:32:17.113123 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.113089    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad25a6b3-6570-43d7-b14a-e035ceb1808b-maas-api-tls\") pod \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\" (UID: \"ad25a6b3-6570-43d7-b14a-e035ceb1808b\") "
Sep 18 18:32:17.113406 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.113380    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad25a6b3-6570-43d7-b14a-e035ceb1808b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ad25a6b3-6570-43d7-b14a-e035ceb1808b" (UID: "ad25a6b3-6570-43d7-b14a-e035ceb1808b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:32:17.113945 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.113913    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad25a6b3-6570-43d7-b14a-e035ceb1808b-openshift-service-ca\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:32:17.115321 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.115301    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad25a6b3-6570-43d7-b14a-e035ceb1808b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ad25a6b3-6570-43d7-b14a-e035ceb1808b" (UID: "ad25a6b3-6570-43d7-b14a-e035ceb1808b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:32:17.115516 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.115500    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad25a6b3-6570-43d7-b14a-e035ceb1808b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ad25a6b3-6570-43d7-b14a-e035ceb1808b" (UID: "ad25a6b3-6570-43d7-b14a-e035ceb1808b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:32:17.115566 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.115526    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad25a6b3-6570-43d7-b14a-e035ceb1808b-kube-api-access-qwcd7" (OuterVolumeSpecName: "kube-api-access-qwcd7") pod "ad25a6b3-6570-43d7-b14a-e035ceb1808b" (UID: "ad25a6b3-6570-43d7-b14a-e035ceb1808b"). InnerVolumeSpecName "kube-api-access-qwcd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:32:17.148116 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.148069    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:17.214840 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.214813    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwcd7\" (UniqueName: \"kubernetes.io/projected/ad25a6b3-6570-43d7-b14a-e035ceb1808b-kube-api-access-qwcd7\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:32:17.214840 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.214838    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad25a6b3-6570-43d7-b14a-e035ceb1808b-metrics-certs\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:32:17.215035 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.214849    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad25a6b3-6570-43d7-b14a-e035ceb1808b-maas-api-tls\") on node \"ip-10-0-130-66.ec2.internal\" DevicePath \"\""
Sep 18 18:32:17.382643 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.382611    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"]
Sep 18 18:32:17.386471 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.386443    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7dddcbdd5b-zgbjc"]
Sep 18 18:32:17.423892 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:17.423869    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"]
Sep 18 18:32:17.425778 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:32:17.425753    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd6521ee_3ddc_4cbd_bd8e_b86020aa9bd6.slice/crio-131bfc05dbfa4bd38ef0764f47e5391abeb77c404dbc5df94cda3eb0709870ab WatchSource:0}: Error finding container 131bfc05dbfa4bd38ef0764f47e5391abeb77c404dbc5df94cda3eb0709870ab: Status 404 returned error can't find the container with id 131bfc05dbfa4bd38ef0764f47e5391abeb77c404dbc5df94cda3eb0709870ab
Sep 18 18:32:18.049656 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:18.049582    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad25a6b3-6570-43d7-b14a-e035ceb1808b" path="/var/lib/kubelet/pods/ad25a6b3-6570-43d7-b14a-e035ceb1808b/volumes"
Sep 18 18:32:18.068518 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:18.068488    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t" event={"ID":"bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6","Type":"ContainerStarted","Data":"0e9fdc6b78598eab49007d6da7a6ec7e94e27db6a8efb92f1f3c9236fb044c3d"}
Sep 18 18:32:18.068665 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:18.068540    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t" event={"ID":"bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6","Type":"ContainerStarted","Data":"131bfc05dbfa4bd38ef0764f47e5391abeb77c404dbc5df94cda3eb0709870ab"}
Sep 18 18:32:18.068665 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:18.068622    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:18.086851 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:18.086804    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t" podStartSLOduration=1.7051439529999999 podStartE2EDuration="2.086788665s" totalImagesPullingTime="381.644712ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:32:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:32:18.084829656 +0000 UTC m=+2542.162633555" watchObservedRunningTime="2026-09-18 18:32:18.086788665 +0000 UTC m=+2542.164592623"
Sep 18 18:32:23.535489 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:23.535454    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:32:24.079153 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:24.079126    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-58fcc54d4-xb28t"
Sep 18 18:32:29.419600 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:29.419563    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:32:36.347425 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:36.347394    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qdzp8"]
Sep 18 18:32:39.023853 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:39.023821    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d8cf95788-vf7vq_135f6fc3-f4fe-4cde-a90e-07ee85823a54/authorino/0.log"
Sep 18 18:32:44.082379 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:44.082346    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-w9zjr_bdfec0cd-ab90-4bea-b28a-b4adbdc15a16/manager/0.log"
Sep 18 18:32:44.191868 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:44.191833    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pnlsz_36258bba-ac53-45cd-b05c-248d4f74caf7/manager/0.log"
Sep 18 18:32:44.762919 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:44.762893    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-649dc7b766-xkzzk_0c818f22-b7e4-4cc2-9fb9-58fba431acd8/copy-manifests/0.log"
Sep 18 18:32:44.769818 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:44.769798    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-649dc7b766-xkzzk_0c818f22-b7e4-4cc2-9fb9-58fba431acd8/manager/0.log"
Sep 18 18:32:45.320439 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:45.320412    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-58fcc54d4-xb28t_bd6521ee-3ddc-4cbd-bd8e-b86020aa9bd6/maas-api/0.log"
Sep 18 18:32:45.427239 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:45.427211    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29829255-l2xvh_96e40a64-5ad0-4e18-b1e3-fac7ce4673a3/cleanup/0.log"
Sep 18 18:32:45.535648 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:45.535620    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29829270-nfgcp_c074e408-98ce-4aa9-800a-f623990faea1/cleanup/0.log"
Sep 18 18:32:46.378625 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:46.378598    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d8cf95788-vf7vq_135f6fc3-f4fe-4cde-a90e-07ee85823a54/authorino/0.log"
Sep 18 18:32:46.490217 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:46.490164    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-7mkn8_f7e509c0-a896-4ebf-89f5-e510eb9f8796/manager/0.log"
Sep 18 18:32:46.593460 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:46.593429    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-kgrb2_77d85aaf-ca7c-4590-b343-4aec7e982f47/manager/0.log"
Sep 18 18:32:46.698562 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:46.698536    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-5r8g9_a95f0f33-d61f-449f-9d38-a103d5ec4709/kuadrant-console-plugin/0.log"
Sep 18 18:32:46.817058 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:46.817028    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-fq2mb_d412c9ec-3863-4e69-8d05-db156736d3d8/manager/0.log"
Sep 18 18:32:46.945844 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:46.945816    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-qdzp8_36d7e60f-5cd0-4535-a69e-712be4a4a58b/limitador/0.log"
Sep 18 18:32:47.068244 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:47.068163    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-rl2wg_952b6ec8-f905-4d6c-905f-9c2844352125/manager/0.log"
Sep 18 18:32:47.634487 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:47.634461    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b7b7f7d54-5mfln_6a06b09b-0510-4c27-8ee0-182b172f05b1/kube-auth-proxy/0.log"
Sep 18 18:32:48.495832 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:48.495809    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-5q5z4_1296604b-af3b-4231-8180-efb81ad8fb9c/main/0.log"
Sep 18 18:32:48.815167 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:48.815087    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-k46b9_4edcbe87-12d7-4731-858d-eb843d1d1c71/main/0.log"
Sep 18 18:32:49.024798 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:49.024771    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-kz7vn_145faaf8-b9ee-427c-9253-a3b13e561ac7/main/0.log"
Sep 18 18:32:57.752611 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:57.752582    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-66bl2_a5aed5d3-d115-484b-a2f9-54a887906baa/global-pull-secret-syncer/0.log"
Sep 18 18:32:57.893635 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:57.893608    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-n6fx7_bd78b5f9-264b-477b-8deb-b91499c6a278/konnectivity-agent/0.log"
Sep 18 18:32:57.970860 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:32:57.970830    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-66.ec2.internal_ac81c3c3c16cb65d907a4298fd5e63bf/haproxy/0.log"
Sep 18 18:33:02.688451 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:02.688412    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d8cf95788-vf7vq_135f6fc3-f4fe-4cde-a90e-07ee85823a54/authorino/0.log"
Sep 18 18:33:02.719842 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:02.719808    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-7mkn8_f7e509c0-a896-4ebf-89f5-e510eb9f8796/manager/0.log"
Sep 18 18:33:02.749368 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:02.749340    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-kgrb2_77d85aaf-ca7c-4590-b343-4aec7e982f47/manager/0.log"
Sep 18 18:33:02.776671 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:02.776637    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-5r8g9_a95f0f33-d61f-449f-9d38-a103d5ec4709/kuadrant-console-plugin/0.log"
Sep 18 18:33:02.817037 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:02.817002    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-fq2mb_d412c9ec-3863-4e69-8d05-db156736d3d8/manager/0.log"
Sep 18 18:33:02.852497 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:02.852469    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-qdzp8_36d7e60f-5cd0-4535-a69e-712be4a4a58b/limitador/0.log"
Sep 18 18:33:02.918257 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:02.918223    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-rl2wg_952b6ec8-f905-4d6c-905f-9c2844352125/manager/0.log"
Sep 18 18:33:04.387029 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:04.387002    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1eb85964-aedf-40b4-bc35-c0ac16d68c10/alertmanager/0.log"
Sep 18 18:33:04.417338 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:04.417310    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1eb85964-aedf-40b4-bc35-c0ac16d68c10/config-reloader/0.log"
Sep 18 18:33:04.440375 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:04.440351    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1eb85964-aedf-40b4-bc35-c0ac16d68c10/kube-rbac-proxy-web/0.log"
Sep 18 18:33:04.468346 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:04.468324    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1eb85964-aedf-40b4-bc35-c0ac16d68c10/kube-rbac-proxy/0.log"
Sep 18 18:33:04.490441 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:04.490415    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1eb85964-aedf-40b4-bc35-c0ac16d68c10/kube-rbac-proxy-metric/0.log"
Sep 18 18:33:04.516083 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:04.516060    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1eb85964-aedf-40b4-bc35-c0ac16d68c10/prom-label-proxy/0.log"
Sep 18 18:33:04.539931 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:04.539907    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1eb85964-aedf-40b4-bc35-c0ac16d68c10/init-config-reloader/0.log"
Sep 18 18:33:04.954995 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:04.954969    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9qbhv_7b29ebc8-1f18-4d33-be23-75eec038322b/node-exporter/0.log"
Sep 18 18:33:04.995143 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:04.995114    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9qbhv_7b29ebc8-1f18-4d33-be23-75eec038322b/kube-rbac-proxy/0.log"
Sep 18 18:33:05.038998 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:05.038871    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9qbhv_7b29ebc8-1f18-4d33-be23-75eec038322b/init-textfile/0.log"
Sep 18 18:33:05.563970 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:05.563939    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-hs89z_7bafd0b2-409e-46ba-a0b8-9391a248ab9f/prometheus-operator/0.log"
Sep 18 18:33:05.593117 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:05.593093    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-hs89z_7bafd0b2-409e-46ba-a0b8-9391a248ab9f/kube-rbac-proxy/0.log"
Sep 18 18:33:05.649800 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:05.649775    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8646db9fbf-x72lb_719fcb78-5c8d-4a1f-811b-2d899e76c7b1/telemeter-client/0.log"
Sep 18 18:33:05.674670 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:05.674645    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8646db9fbf-x72lb_719fcb78-5c8d-4a1f-811b-2d899e76c7b1/reload/0.log"
Sep 18 18:33:05.695886 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:05.695862    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8646db9fbf-x72lb_719fcb78-5c8d-4a1f-811b-2d899e76c7b1/kube-rbac-proxy/0.log"
Sep 18 18:33:05.723839 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:05.723819    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b6cb5d5df-mpktj_da6ebfa0-2312-4815-bc5a-a45adafdd82a/thanos-query/0.log"
Sep 18 18:33:05.746087 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:05.746067    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b6cb5d5df-mpktj_da6ebfa0-2312-4815-bc5a-a45adafdd82a/kube-rbac-proxy-web/0.log"
Sep 18 18:33:05.769547 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:05.769524    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b6cb5d5df-mpktj_da6ebfa0-2312-4815-bc5a-a45adafdd82a/kube-rbac-proxy/0.log"
Sep 18 18:33:05.791245 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:05.791219    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b6cb5d5df-mpktj_da6ebfa0-2312-4815-bc5a-a45adafdd82a/prom-label-proxy/0.log"
Sep 18 18:33:05.811713 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:05.811694    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b6cb5d5df-mpktj_da6ebfa0-2312-4815-bc5a-a45adafdd82a/kube-rbac-proxy-rules/0.log"
Sep 18 18:33:05.832060 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:05.832009    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7b6cb5d5df-mpktj_da6ebfa0-2312-4815-bc5a-a45adafdd82a/kube-rbac-proxy-metrics/0.log"
Sep 18 18:33:07.842419 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:07.842383    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"]
Sep 18 18:33:07.842797 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:07.842779    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad25a6b3-6570-43d7-b14a-e035ceb1808b" containerName="maas-api"
Sep 18 18:33:07.842797 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:07.842792    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad25a6b3-6570-43d7-b14a-e035ceb1808b" containerName="maas-api"
Sep 18 18:33:07.842882 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:07.842872    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad25a6b3-6570-43d7-b14a-e035ceb1808b" containerName="maas-api"
Sep 18 18:33:07.847089 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:07.847073    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:07.849293 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:07.849274    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tkl5n\"/\"default-dockercfg-btmrw\""
Sep 18 18:33:07.849581 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:07.849565    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tkl5n\"/\"kube-root-ca.crt\""
Sep 18 18:33:07.849673 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:07.849581    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tkl5n\"/\"openshift-service-ca.crt\""
Sep 18 18:33:07.855017 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:07.855000    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"]
Sep 18 18:33:07.963265 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:07.963234    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d371ef4a-7a74-482f-bf04-187629064db5-lib-modules\") pod \"perf-node-gather-daemonset-nmcpp\" (UID: \"d371ef4a-7a74-482f-bf04-187629064db5\") " pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:07.963414 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:07.963279    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb29t\" (UniqueName: \"kubernetes.io/projected/d371ef4a-7a74-482f-bf04-187629064db5-kube-api-access-nb29t\") pod \"perf-node-gather-daemonset-nmcpp\" (UID: \"d371ef4a-7a74-482f-bf04-187629064db5\") " pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:07.963414 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:07.963362    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d371ef4a-7a74-482f-bf04-187629064db5-proc\") pod \"perf-node-gather-daemonset-nmcpp\" (UID: \"d371ef4a-7a74-482f-bf04-187629064db5\") " pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:07.963414 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:07.963397    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d371ef4a-7a74-482f-bf04-187629064db5-sys\") pod \"perf-node-gather-daemonset-nmcpp\" (UID: \"d371ef4a-7a74-482f-bf04-187629064db5\") " pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:07.963525 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:07.963426    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d371ef4a-7a74-482f-bf04-187629064db5-podres\") pod \"perf-node-gather-daemonset-nmcpp\" (UID: \"d371ef4a-7a74-482f-bf04-187629064db5\") " pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:08.063870 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:08.063840    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d371ef4a-7a74-482f-bf04-187629064db5-lib-modules\") pod \"perf-node-gather-daemonset-nmcpp\" (UID: \"d371ef4a-7a74-482f-bf04-187629064db5\") " pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:08.064045 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:08.063880    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb29t\" (UniqueName: \"kubernetes.io/projected/d371ef4a-7a74-482f-bf04-187629064db5-kube-api-access-nb29t\") pod \"perf-node-gather-daemonset-nmcpp\" (UID: \"d371ef4a-7a74-482f-bf04-187629064db5\") " pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:08.064045 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:08.063905    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d371ef4a-7a74-482f-bf04-187629064db5-proc\") pod \"perf-node-gather-daemonset-nmcpp\" (UID: \"d371ef4a-7a74-482f-bf04-187629064db5\") " pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:08.064045 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:08.063920    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d371ef4a-7a74-482f-bf04-187629064db5-sys\") pod \"perf-node-gather-daemonset-nmcpp\" (UID: \"d371ef4a-7a74-482f-bf04-187629064db5\") " pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:08.064045 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:08.063945    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d371ef4a-7a74-482f-bf04-187629064db5-podres\") pod \"perf-node-gather-daemonset-nmcpp\" (UID: \"d371ef4a-7a74-482f-bf04-187629064db5\") " pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:08.064045 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:08.063976    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d371ef4a-7a74-482f-bf04-187629064db5-proc\") pod \"perf-node-gather-daemonset-nmcpp\" (UID: \"d371ef4a-7a74-482f-bf04-187629064db5\") " pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:08.064045 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:08.064001    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d371ef4a-7a74-482f-bf04-187629064db5-sys\") pod \"perf-node-gather-daemonset-nmcpp\" (UID: \"d371ef4a-7a74-482f-bf04-187629064db5\") " pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:08.064045 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:08.064007    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d371ef4a-7a74-482f-bf04-187629064db5-lib-modules\") pod \"perf-node-gather-daemonset-nmcpp\" (UID: \"d371ef4a-7a74-482f-bf04-187629064db5\") " pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:08.064045 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:08.064048    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d371ef4a-7a74-482f-bf04-187629064db5-podres\") pod \"perf-node-gather-daemonset-nmcpp\" (UID: \"d371ef4a-7a74-482f-bf04-187629064db5\") " pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:08.072914 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:08.072895    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb29t\" (UniqueName: \"kubernetes.io/projected/d371ef4a-7a74-482f-bf04-187629064db5-kube-api-access-nb29t\") pod \"perf-node-gather-daemonset-nmcpp\" (UID: \"d371ef4a-7a74-482f-bf04-187629064db5\") " pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:08.157673 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:08.157650    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:08.349556 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:08.349520    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-bcc9bbb87-kkhlz_98599823-a747-47b5-a3d7-a6a8a9fa6127/console/0.log"
Sep 18 18:33:08.382780 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:08.382747    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-sj7s5_408c606b-e5cc-4afc-9a3d-afcab63ba455/download-server/0.log"
Sep 18 18:33:08.426820 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:08.426788    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"]
Sep 18 18:33:08.428629 ip-10-0-130-66 kubenswrapper[2568]: W0918 18:33:08.428604    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd371ef4a_7a74_482f_bf04_187629064db5.slice/crio-4532fbba3354f105e10c9917cda15a49e63128b24d2fe17be39ae1923c1af7fb WatchSource:0}: Error finding container 4532fbba3354f105e10c9917cda15a49e63128b24d2fe17be39ae1923c1af7fb: Status 404 returned error can't find the container with id 4532fbba3354f105e10c9917cda15a49e63128b24d2fe17be39ae1923c1af7fb
Sep 18 18:33:09.335738 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:09.335704    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp" event={"ID":"d371ef4a-7a74-482f-bf04-187629064db5","Type":"ContainerStarted","Data":"bc8fb38651c818e4e098385f6568b629ca5d71d30768c6ef099a849d70dca4ff"}
Sep 18 18:33:09.336132 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:09.335758    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp" event={"ID":"d371ef4a-7a74-482f-bf04-187629064db5","Type":"ContainerStarted","Data":"4532fbba3354f105e10c9917cda15a49e63128b24d2fe17be39ae1923c1af7fb"}
Sep 18 18:33:09.336132 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:09.335820    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:09.354031 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:09.353979    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp" podStartSLOduration=2.353961972 podStartE2EDuration="2.353961972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:33:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:33:09.352609028 +0000 UTC m=+2593.430412927" watchObservedRunningTime="2026-09-18 18:33:09.353961972 +0000 UTC m=+2593.431765871"
Sep 18 18:33:10.409266 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:10.409231    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gtq6p_96a387f5-7cde-4173-9023-721818abe42f/dns/0.log"
Sep 18 18:33:10.446963 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:10.446936    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gtq6p_96a387f5-7cde-4173-9023-721818abe42f/kube-rbac-proxy/0.log"
Sep 18 18:33:10.617368 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:10.617302    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t457f_85ae4e18-9ca4-44dc-9009-4c8d9e4c82e1/dns-node-resolver/0.log"
Sep 18 18:33:11.221497 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:11.221466    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-9568558f4-kz48x_46939250-033d-42f7-a8b8-1330f875a69b/registry/0.log"
Sep 18 18:33:11.283746 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:11.283718    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nhbw2_968ec864-e2d4-4577-9a8e-4e63b405acfd/node-ca/0.log"
Sep 18 18:33:13.123581 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:13.123555    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b7b7f7d54-5mfln_6a06b09b-0510-4c27-8ee0-182b172f05b1/kube-auth-proxy/0.log"
Sep 18 18:33:14.377912 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:14.377881    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mkzvh_00e9de95-8e8f-4141-8173-8521477998f8/serve-healthcheck-canary/0.log"
Sep 18 18:33:15.352519 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:15.352495    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tkl5n/perf-node-gather-daemonset-nmcpp"
Sep 18 18:33:15.376851 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:15.376825    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qmz9s_113e5dce-2f88-4605-b681-086bcb92bb80/kube-rbac-proxy/0.log"
Sep 18 18:33:15.445824 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:15.445796    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qmz9s_113e5dce-2f88-4605-b681-086bcb92bb80/exporter/0.log"
Sep 18 18:33:15.512333 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:15.512303    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qmz9s_113e5dce-2f88-4605-b681-086bcb92bb80/extractor/0.log"
Sep 18 18:33:18.076274 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:18.076243    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-w9zjr_bdfec0cd-ab90-4bea-b28a-b4adbdc15a16/manager/0.log"
Sep 18 18:33:18.363735 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:18.363640    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pnlsz_36258bba-ac53-45cd-b05c-248d4f74caf7/manager/0.log"
Sep 18 18:33:18.968609 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:18.968570    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b7b7f7d54-5mfln_6a06b09b-0510-4c27-8ee0-182b172f05b1/kube-auth-proxy/0.log"
Sep 18 18:33:19.270678 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:19.270598    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-649dc7b766-xkzzk_0c818f22-b7e4-4cc2-9fb9-58fba431acd8/manager/0.log"
Sep 18 18:33:19.338818 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:19.338792    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-649dc7b766-xkzzk_0c818f22-b7e4-4cc2-9fb9-58fba431acd8/copy-manifests/0.log"
Sep 18 18:33:21.328315 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:21.328283    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-tfbmp_a9d303be-d7c5-4ec2-87cb-318bb40b5e46/openshift-lws-operator/0.log"
Sep 18 18:33:21.588368 ip-10-0-130-66 kubenswrapper[2568]: I0918 18:33:21.588291    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b7b7f7d54-5mfln_6a06b09b-0510-4c27-8ee0-182b172f05b1/kube-auth-proxy/0.log"