Sep 11 13:21:19.222908 ip-10-0-142-132 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 13:21:19.696439 ip-10-0-142-132 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:21:19.696439 ip-10-0-142-132 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 13:21:19.696439 ip-10-0-142-132 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:21:19.696439 ip-10-0-142-132 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 13:21:19.696439 ip-10-0-142-132 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:21:19.698878 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.698786    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 13:21:19.703394 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703377    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:21:19.703394 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703394    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703398    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703402    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703405    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703408    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703411    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703414    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703417    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703421    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703423    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703426    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703429    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703432    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703435    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703438    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703441    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703444    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703447    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703449    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703452    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:21:19.703459 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703455    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703457    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703460    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703465    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703469    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703472    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703475    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703478    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703480    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703483    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703485    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703488    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703491    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703493    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703496    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703498    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703501    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703504    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703506    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:21:19.703951 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703509    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703511    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703514    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703517    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703519    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703522    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703524    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703528    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703531    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703533    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703536    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703538    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703540    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703543    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703545    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703549    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703552    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703554    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703557    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:21:19.704455 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703560    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703562    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703565    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703567    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703572    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703575    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703578    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703581    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703583    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703586    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703589    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703606    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703610    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703614    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703617    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703619    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703622    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703625    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703627    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:21:19.704928 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703630    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703632    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703635    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703638    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703640    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703642    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703645    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703648    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703650    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703653    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703658    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703661    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703664    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703666    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703669    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703672    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703675    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703677    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703680    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703682    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:21:19.705384 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.703685    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704467    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704474    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704477    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704480    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704483    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704486    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704489    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704492    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704495    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704498    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704501    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704504    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704506    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704509    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704511    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704514    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704516    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704519    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704522    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:21:19.705911 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704524    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704527    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704530    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704532    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704535    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704538    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704541    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704543    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704546    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704548    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704551    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704554    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704556    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704559    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704561    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704565    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704568    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704571    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704574    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704577    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:21:19.706407 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704580    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704582    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704584    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704587    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704590    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704608    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704612    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704614    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704617    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704620    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704622    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704625    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704627    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704630    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704633    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704635    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704638    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704641    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704643    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704646    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704648    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:21:19.706947 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704651    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704655    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704658    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704661    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704665    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704669    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704672    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704674    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704677    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704682    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704685    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704688    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704690    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704693    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704696    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704698    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704701    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704703    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:21:19.707463 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704706    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704709    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704712    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704714    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704717    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704719    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704722    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704724    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704727    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704730    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704732    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704735    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704737    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704740    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704743    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704745    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704748    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704750    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704753    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704755    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:21:19.707929 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.704758    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705116    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705121    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705125    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705128    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705136    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705139    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705142    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705144    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705147    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705150    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705152    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705155    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705158    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705161    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705164    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705166    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705170    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705173    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705175    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:21:19.708413 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705178    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705181    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705184    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705188    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705191    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705194    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705196    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705199    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705201    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705206    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705209    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705212    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705215    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705218    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705221    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705225    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705227    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705230    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705239    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:21:19.708908 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705241    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705244    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705246    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705249    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705251    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705254    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705256    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705260    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705263    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705266    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705269    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705272    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705275    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705278    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705281    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705284    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705286    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705289    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:21:19.709368 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705292    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705294    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705296    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705299    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705302    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705305    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705308    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705310    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705313    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705315    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705318    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705321    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705324    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705326    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705334    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705337    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705339    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705342    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705345    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705348    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:21:19.709889 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705350    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705353    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705355    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705358    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705360    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705363    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705366    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705368    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705372    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705374    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705377    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705379    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705381    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705384    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705387    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705390    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705392    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705395    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705398    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705400    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:21:19.710382 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705403    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705405    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.705408    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705496    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705536    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705548    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705552    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705557    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705565    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705574    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705578    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705582    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705585    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705589    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705603    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705607    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705610    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705613    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705616    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705619    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705622    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705629    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705633    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705636    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 13:21:19.710879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705639    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705642    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705646    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705649    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705653    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705656    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705659    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705669    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705673    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705676    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705682    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705687    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705690    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705693    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705696    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705699    2580 flags.go:64] FLAG: --enable-server="true"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705702    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705709    2580 flags.go:64] FLAG: --event-burst="100"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705712    2580 flags.go:64] FLAG: --event-qps="50"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705716    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705719    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705722    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705726    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705729    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705732    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 13:21:19.711505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705735    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705738    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705741    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705746    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705748    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705751    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705754    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705757    2580 flags.go:64] FLAG: --feature-gates=""
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705760    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705764    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705767    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705770    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705773    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705776    2580 flags.go:64] FLAG: --help="false"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705779    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-142-132.ec2.internal"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705782    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705785    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705789    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705792    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705795    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705798    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705801    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705804    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 13:21:19.712168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705807    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705811    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705815    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705821    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705824    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705827    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705830    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705833    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705836    2580 flags.go:64] FLAG: --lock-file=""
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705838    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705841    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705844    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705850    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705853    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705856    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705858    2580 flags.go:64] FLAG: --logging-format="text"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705861    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705864    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705867    2580 flags.go:64] FLAG: --manifest-url=""
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705870    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705875    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705877    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705881    2580 flags.go:64] FLAG: --max-pods="110"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705884    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705887    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 13:21:19.712816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705890    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705896    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705899    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705902    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705905    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705912    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705915    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705918    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705923    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705926    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705932    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705935    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705938    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705941    2580 flags.go:64] FLAG: --port="10250"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705944    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705947    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0277e4c66bf9ebbf8"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705950    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705953    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705956    2580 flags.go:64] FLAG: --register-node="true"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705959    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705962    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705965    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705968    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705971    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705974    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 13:21:19.713440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705977    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705980    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705983    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705986    2580 flags.go:64] FLAG: --runonce="false"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705989    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705992    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705995    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.705998    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706002    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706005    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706008    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706011    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706014    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706016    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706019    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706022    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706027    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706030    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706033    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706039    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706041    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706045    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706048    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706051    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706054    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 13:21:19.714084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706057    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706060    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706063    2580 flags.go:64] FLAG: --v="2"
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706068    2580 flags.go:64] FLAG: --version="false"
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706072    2580 flags.go:64] FLAG: --vmodule=""
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706075    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.706079    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706210    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706214    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706217    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706220    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706223    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706225    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706228    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706231    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706235    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706238    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706240    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706243    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706246    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706248    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:21:19.714710 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706250    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706253    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706256    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706259    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706262    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706265    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706267    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706271    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706273    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706276    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706279    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706281    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706284    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706286    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706289    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706293    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706297    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706300    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706303    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706307    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:21:19.715217 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706310    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706312    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706314    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706317    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706319    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706322    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706326    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706328    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706331    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706335    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706338    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706341    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706344    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706346    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706350    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706352    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706355    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706358    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706360    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:21:19.715747 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706363    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706366    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706368    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706371    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706373    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706376    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706378    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706381    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706384    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706386    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706389    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706391    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706394    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706396    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706399    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706401    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706404    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706406    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706409    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706413    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:21:19.716205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706416    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706419    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706422    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706424    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706427    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706430    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706432    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706436    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706438    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706441    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706444    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706447    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706450    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706452    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706455    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706458    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706461    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706464    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706466    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706469    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:21:19.716717 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706472    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:21:19.717205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706475    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:21:19.717205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706479    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:21:19.717205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706482    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:21:19.717205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706485    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:21:19.717205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.706487    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:21:19.717205 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.707231    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:21:19.717205 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.715150    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 13:21:19.717205 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.715164    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 13:21:19.717205 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.715898    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 13:21:19.717205 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.715906    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 13:21:19.717205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.715971    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:21:19.717205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.715977    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:21:19.717205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.715981    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:21:19.717205 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.715984    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.715987    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.715990    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.715992    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.715995    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.715998    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716000    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716003    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716006    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716014    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716019    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716023    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716026    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716028    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716031    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716033    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716036    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716039    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716041    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:21:19.717550 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716044    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716047    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716050    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716052    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716055    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716057    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716060    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716064    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716066    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716069    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716071    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716074    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716077    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716080    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716082    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716085    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716088    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716090    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716094    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:21:19.718032 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716098    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716100    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716103    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716106    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716115    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716118    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716121    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716124    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716127    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716129    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716132    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716135    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716143    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716146    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716148    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716152    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716154    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716156    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716159    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716162    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:21:19.718494 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716165    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716169    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716173    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716176    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716178    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716181    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716184    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716186    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716189    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716192    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716194    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716197    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716199    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716202    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716204    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716207    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716209    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716218    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716221    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716224    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:21:19.718993 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716226    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716229    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716231    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716234    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716237    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716239    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716242    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716244    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716247    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716250    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716252    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716255    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716258    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716260    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716263    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716266    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716268    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:21:19.719477 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716270    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:21:19.719904 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.716276    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:21:19.719904 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716446    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:21:19.719904 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716452    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:21:19.719904 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716456    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:21:19.719904 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716459    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:21:19.719904 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716461    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:21:19.719904 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716464    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:21:19.719904 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716466    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:21:19.719904 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716469    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:21:19.719904 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716472    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:21:19.719904 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716475    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:21:19.719904 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716477    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:21:19.719904 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716486    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:21:19.719904 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716488    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716491    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716493    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716496    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716499    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716503    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716506    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716509    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716512    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716515    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716517    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716520    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716522    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716525    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716527    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716530    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716533    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716535    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716538    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:21:19.720241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716540    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716543    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716545    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716548    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716550    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716553    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716555    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716558    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716560    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716563    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716565    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716568    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716570    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716579    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716581    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716584    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716586    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716589    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716606    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716610    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716612    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:21:19.720715 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716615    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716617    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716620    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716623    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716625    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716628    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716630    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716633    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716635    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716638    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716640    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716643    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716645    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716648    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716652    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716655    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716658    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716661    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716663    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:21:19.721241 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716666    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716669    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716672    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716675    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716678    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716680    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716689    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716692    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716695    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716697    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716700    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716702    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716704    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716707    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716710    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716713    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716715    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716718    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716721    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716723    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:21:19.721783 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716726    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:21:19.722261 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716729    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:21:19.722261 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716731    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:21:19.722261 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716734    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:21:19.722261 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716736    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:21:19.722261 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716739    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:21:19.722261 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716741    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:21:19.722261 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:19.716744    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:21:19.722261 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.716749    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:21:19.722261 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.716859    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 13:21:19.722261 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.719697    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 13:21:19.722261 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.721252    2580 server.go:1019] "Starting client certificate rotation"
Sep 11 13:21:19.722261 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.721384    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:21:19.722261 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.721418    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:21:19.724437 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.724423    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:21:19.729203 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.729179    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:21:19.743161 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.743141    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 11 13:21:19.746584 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.746566    2580 log.go:25] "Validated CRI v1 image API"
Sep 11 13:21:19.747414 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.747394    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 13:21:19.750152 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.750126    2580 fs.go:135] Filesystem UUIDs: map[07e32d56-a61f-4334-8236-49abf1e26cf6:/dev/nvme0n1p4 3fbe2e8c-8fa2-4898-a468-e0d13c8fd17d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 11 13:21:19.750205 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.750152    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 13:21:19.755074 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.754955    2580 manager.go:216] Machine: {Timestamp:2026-09-11 13:21:19.753825015 +0000 UTC m=+0.391669928 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:2943652 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec207f456769aa0466be2f2e49c37c37 SystemUUID:ec207f45-6769-aa04-66be-2f2e49c37c37 BootID:2d1fa8b7-99fc-4f24-8f55-80cd1acd35ac Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:02:6a:17:4d:e5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:02:6a:17:4d:e5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:77:7f:85:1f:da Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 13:21:19.755074 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.755071    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 13:21:19.755197 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.755156    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 13:21:19.757508 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.757484    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:21:19.757979 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.757947    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 13:21:19.758118 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.757982    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-132.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 13:21:19.758622 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.758613    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 13:21:19.758662 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.758626    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 13:21:19.758662 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.758633    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:21:19.760027 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.760014    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:21:19.760735 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.760725    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 13:21:19.766993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.766980    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:21:19.767124 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.767115    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 13:21:19.767154 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.767148    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 13:21:19.767185 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.767158    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 13:21:19.767185 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.767179    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 13:21:19.767239 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.767221    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 11 13:21:19.767239 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.767230    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 13:21:19.767794 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.767782    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:21:19.767836 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.767803    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:21:19.770022 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.769920    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 13:21:19.771964 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.771950    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 13:21:19.772014 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.771985    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 13:21:19.774653 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.774635    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 13:21:19.774688 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.774658    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 13:21:19.774688 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.774665    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 13:21:19.774688 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.774671    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 13:21:19.774688 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.774684    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 13:21:19.774790 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.774692    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 13:21:19.774790 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.774705    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 13:21:19.774790 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.774711    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 13:21:19.774790 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.774728    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 13:21:19.774790 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.774736    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 13:21:19.774790 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.774750    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 13:21:19.774790 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.774762    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 13:21:19.776536 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.776522    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 13:21:19.776536 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.776533    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 13:21:19.778676 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:19.778649    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 13:21:19.778978 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.778942    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-132.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 13:21:19.779123 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:19.779090    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-132.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 13:21:19.780746 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.780728    2580 server.go:1268] "Started kubelet"
Sep 11 13:21:19.781060 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.781028    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 13:21:19.781189 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.780992    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 13:21:19.781364 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.781347    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 13:21:19.781790 ip-10-0-142-132 systemd[1]: Started Kubernetes Kubelet.
Sep 11 13:21:19.783916 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.783728    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 13:21:19.784451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.784303    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 13:21:19.789103 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:19.789083    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 13:21:19.790371 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.790357    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tsgcn"
Sep 11 13:21:19.791847 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.791831    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 13:21:19.792427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.792408    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 13:21:19.792427 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:19.791433    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-132.ec2.internal.18d446dffb4ccb69  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-132.ec2.internal,UID:ip-10-0-142-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-132.ec2.internal,},FirstTimestamp:2026-09-11 13:21:19.780694889 +0000 UTC m=+0.418539806,LastTimestamp:2026-09-11 13:21:19.780694889 +0000 UTC m=+0.418539806,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-132.ec2.internal,}"
Sep 11 13:21:19.792564 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:19.792550    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-132.ec2.internal\" not found"
Sep 11 13:21:19.793245 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.793151    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 13:21:19.793245 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.793197    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 13:21:19.793245 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.793212    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 13:21:19.793570 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.793262    2580 factory.go:55] Registering systemd factory
Sep 11 13:21:19.793570 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.793277    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 11 13:21:19.793570 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.793292    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 13:21:19.793570 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.793303    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 13:21:19.793570 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.793517    2580 factory.go:153] Registering CRI-O factory
Sep 11 13:21:19.793570 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.793530    2580 factory.go:223] Registration of the crio container factory successfully
Sep 11 13:21:19.793843 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.793580    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 13:21:19.793843 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.793614    2580 factory.go:103] Registering Raw factory
Sep 11 13:21:19.793843 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.793631    2580 manager.go:1196] Started watching for new ooms in manager
Sep 11 13:21:19.794072 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.794057    2580 manager.go:349] Starting recovery of all containers
Sep 11 13:21:19.804398 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.804350    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tsgcn"
Sep 11 13:21:19.806974 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.806960    2580 manager.go:354] Recovery completed
Sep 11 13:21:19.808896 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:19.808875    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 13:21:19.809538 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:19.809513    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-132.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 13:21:19.811854 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.811837    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:21:19.813531 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.813509    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:21:19.813622 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.813543    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:21:19.813622 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.813554    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:21:19.814296 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.814283    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 13:21:19.814296 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.814296    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 13:21:19.814419 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.814298    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:21:19.814419 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.814313    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:21:19.815091 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.815076    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:21:19.815157 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.815102    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:21:19.815157 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.815111    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:21:19.815414 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.815401    2580 policy_none.go:49] "None policy: Start"
Sep 11 13:21:19.815467 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.815421    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 13:21:19.815467 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.815437    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 13:21:19.816396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.816383    2580 policy_none.go:47] "Start"
Sep 11 13:21:19.862209 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.862192    2580 manager.go:337] "Starting Device Plugin manager"
Sep 11 13:21:19.862355 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:19.862233    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 13:21:19.862355 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.862244    2580 server.go:87] "Starting device plugin registration server"
Sep 11 13:21:19.862705 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.862691    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 13:21:19.862751 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.862708    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 13:21:19.862855 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.862831    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 13:21:19.862961 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.862928    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 13:21:19.862961 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.862938    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 13:21:19.863642 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:19.863625    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 13:21:19.863736 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:19.863678    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-132.ec2.internal\" not found"
Sep 11 13:21:19.888257 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.888226    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 13:21:19.889421 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.889402    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 13:21:19.889510 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.889462    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 13:21:19.889510 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.889490    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 13:21:19.889618 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:19.889587    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 13:21:19.892021 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.892004    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:21:19.963384 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.963280    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:21:19.964282 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.964258    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:21:19.964399 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.964298    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:21:19.964399 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.964309    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:21:19.964399 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.964333    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-132.ec2.internal"
Sep 11 13:21:19.978761 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.978739    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-132.ec2.internal"
Sep 11 13:21:19.978883 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:19.978767    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-132.ec2.internal\": node \"ip-10-0-142-132.ec2.internal\" not found"
Sep 11 13:21:19.990699 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.990665    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-132.ec2.internal"]
Sep 11 13:21:19.990763 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.990748    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:21:19.991706 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.991691    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:21:19.991775 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.991718    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:21:19.991775 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.991728    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:21:19.992927 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.992914    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:21:19.993067 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.993052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal"
Sep 11 13:21:19.993124 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.993090    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:21:19.993647 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.993634    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:21:19.993647 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.993640    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:21:19.993779 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.993659    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:21:19.993779 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.993670    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:21:19.993779 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.993680    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:21:19.993779 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.993686    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:21:19.994802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.994782    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-132.ec2.internal"
Sep 11 13:21:19.994898 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.994813    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:21:19.994898 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.994861    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/102eeadc7fecd8005cb07392235d7fc0-config\") pod \"kube-apiserver-proxy-ip-10-0-142-132.ec2.internal\" (UID: \"102eeadc7fecd8005cb07392235d7fc0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-132.ec2.internal"
Sep 11 13:21:19.994898 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.994891    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a39b3bf3052335ff9b476d16bbcf3e94-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal\" (UID: \"a39b3bf3052335ff9b476d16bbcf3e94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal"
Sep 11 13:21:19.995097 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.994920    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a39b3bf3052335ff9b476d16bbcf3e94-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal\" (UID: \"a39b3bf3052335ff9b476d16bbcf3e94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal"
Sep 11 13:21:19.995547 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.995529    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:21:19.995635 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.995556    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:21:19.995635 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:19.995565    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:21:20.019277 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:20.019250    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-132.ec2.internal\" not found" node="ip-10-0-142-132.ec2.internal"
Sep 11 13:21:20.023549 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:20.023531    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-132.ec2.internal\" not found" node="ip-10-0-142-132.ec2.internal"
Sep 11 13:21:20.095527 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:20.095490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a39b3bf3052335ff9b476d16bbcf3e94-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal\" (UID: \"a39b3bf3052335ff9b476d16bbcf3e94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal"
Sep 11 13:21:20.095527 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:20.095530    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a39b3bf3052335ff9b476d16bbcf3e94-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal\" (UID: \"a39b3bf3052335ff9b476d16bbcf3e94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal"
Sep 11 13:21:20.095756 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:20.095555    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/102eeadc7fecd8005cb07392235d7fc0-config\") pod \"kube-apiserver-proxy-ip-10-0-142-132.ec2.internal\" (UID: \"102eeadc7fecd8005cb07392235d7fc0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-132.ec2.internal"
Sep 11 13:21:20.095756 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:20.095613    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a39b3bf3052335ff9b476d16bbcf3e94-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal\" (UID: \"a39b3bf3052335ff9b476d16bbcf3e94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal"
Sep 11 13:21:20.095756 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:20.095625    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/102eeadc7fecd8005cb07392235d7fc0-config\") pod \"kube-apiserver-proxy-ip-10-0-142-132.ec2.internal\" (UID: \"102eeadc7fecd8005cb07392235d7fc0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-132.ec2.internal"
Sep 11 13:21:20.095756 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:20.095625    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a39b3bf3052335ff9b476d16bbcf3e94-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal\" (UID: \"a39b3bf3052335ff9b476d16bbcf3e94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal"
Sep 11 13:21:20.096750 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:20.096733    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-132.ec2.internal\" not found"
Sep 11 13:21:20.196899 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:20.196861    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-132.ec2.internal\" not found"
Sep 11 13:21:21.717207 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:21.717180    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-132.ec2.internal\" not found"
Sep 11 13:21:21.741348 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:21.741319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal"
Sep 11 13:21:21.745144 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:21.745129    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-132.ec2.internal"
Sep 11 13:21:21.817705 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:21.817667    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-132.ec2.internal\" not found"
Sep 11 13:21:21.918091 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:21.918069    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-132.ec2.internal\" not found"
Sep 11 13:21:22.018662 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:22.018588    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-132.ec2.internal\" not found"
Sep 11 13:21:22.119037 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:22.119006    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-132.ec2.internal\" not found"
Sep 11 13:21:22.140196 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:22.140158    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 13:21:22.140316 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:22.140302    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 13:21:22.219526 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:22.219494    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-132.ec2.internal\" not found"
Sep 11 13:21:22.228029 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:22.227979    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 13:16:21 +0000 UTC" deadline="2028-06-27 20:24:04.66139467 +0000 UTC"
Sep 11 13:21:22.228029 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:22.228026    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15727h2m42.433372257s"
Sep 11 13:21:22.317706 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:22.317675    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod102eeadc7fecd8005cb07392235d7fc0.slice/crio-5287d0891bc597b8e330536f52a7f7eac67420647b2d3aff8278e215433b7427 WatchSource:0}: Error finding container 5287d0891bc597b8e330536f52a7f7eac67420647b2d3aff8278e215433b7427: Status 404 returned error can't find the container with id 5287d0891bc597b8e330536f52a7f7eac67420647b2d3aff8278e215433b7427
Sep 11 13:21:22.319710 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:22.319692    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-132.ec2.internal\" not found"
Sep 11 13:21:22.321829 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:22.321807    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:21:22.419798 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:22.419762    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-132.ec2.internal\" not found"
Sep 11 13:21:22.520216 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:22.520183    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-132.ec2.internal\" not found"
Sep 11 13:21:22.620755 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:22.620696    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-132.ec2.internal\" not found"
Sep 11 13:21:22.638347 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:22.638325    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:21:22.712193 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:22.712167    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal"
Sep 11 13:21:22.727698 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:22.727677    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:21:22.731075 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:22.731056    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-132.ec2.internal"
Sep 11 13:21:22.736554 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:22.736531    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:21:22.744406 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:22.744388    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:21:22.825005 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:22.824827    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:21:23.049417 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.049187    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:21:23.187974 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.187942    2580 apiserver.go:52] "Watching apiserver"
Sep 11 13:21:23.194594 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.194573    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 13:21:23.195569 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.195533    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-2d9t8","openshift-ovn-kubernetes/ovnkube-node-cl86h","kube-system/global-pull-secret-syncer-fld56","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg","openshift-dns/node-resolver-rkrhj","openshift-multus/multus-additional-cni-plugins-7lw4p","openshift-multus/network-metrics-daemon-hxs7m","openshift-network-operator/iptables-alerter-46b7m","kube-system/konnectivity-agent-xbv6v","kube-system/kube-apiserver-proxy-ip-10-0-142-132.ec2.internal","openshift-cluster-node-tuning-operator/tuned-pqdpt","openshift-image-registry/node-ca-gz6tv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal","openshift-multus/multus-l8fqj"]
Sep 11 13:21:23.198012 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.197988    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:23.198207 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:23.198084    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:23.200196 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.200106    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.202317 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.202200    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:23.202317 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:23.202275    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:23.203850 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.203689    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9q8ns\""
Sep 11 13:21:23.203850 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.203717    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 13:21:23.203850 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.203828    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:21:23.204709 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.204689    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.206939 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.206780    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rkrhj"
Sep 11 13:21:23.209260 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.208836    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.209739 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.209720    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 13:21:23.210074 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.210053    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 13:21:23.210200 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.210184    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 13:21:23.210246 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.210056    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 13:21:23.210246 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.210205    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bmpx7\""
Sep 11 13:21:23.210407 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.210341    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 13:21:23.210572 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.210554    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-x4j72\""
Sep 11 13:21:23.211062 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.211044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xbv6v"
Sep 11 13:21:23.212336 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.212319    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 13:21:23.213458 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.213440    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.216051 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.216035    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.218082 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.218062    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 13:21:23.218665 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.218646    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2crdk\""
Sep 11 13:21:23.218838 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.218823    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 13:21:23.220473 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.219054    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 13:21:23.220473 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.219105    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 13:21:23.220473 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.219643    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-46b7m"
Sep 11 13:21:23.222370 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.222318    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:23.222463 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:23.222397    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:23.223822 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.223757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-socket-dir\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.223822 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.223800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-system-cni-dir\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.224077 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.223831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-multus-conf-dir\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.224077 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.223890    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90842e4f-ef4d-490b-84f1-265e5b4d076f-multus-daemon-config\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.224077 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.223940    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/000a37ec-f62c-4808-a1ec-2992d0f919a4-konnectivity-ca\") pod \"konnectivity-agent-xbv6v\" (UID: \"000a37ec-f62c-4808-a1ec-2992d0f919a4\") " pod="kube-system/konnectivity-agent-xbv6v"
Sep 11 13:21:23.224077 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.223967    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-hostroot\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.224077 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:23.224077 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224031    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-cnibin\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.224372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224075    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-device-dir\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.224372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/030910bd-d1c8-48c5-bc44-5e561b3fbf60-hosts-file\") pod \"node-resolver-rkrhj\" (UID: \"030910bd-d1c8-48c5-bc44-5e561b3fbf60\") " pod="openshift-dns/node-resolver-rkrhj"
Sep 11 13:21:23.224372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224151    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-sysconfig\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.224372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/030910bd-d1c8-48c5-bc44-5e561b3fbf60-tmp-dir\") pod \"node-resolver-rkrhj\" (UID: \"030910bd-d1c8-48c5-bc44-5e561b3fbf60\") " pod="openshift-dns/node-resolver-rkrhj"
Sep 11 13:21:23.224372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-system-cni-dir\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.224372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224221    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.224372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224243    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-run\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.224372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmxx6\" (UniqueName: \"kubernetes.io/projected/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-kube-api-access-jmxx6\") pod \"network-metrics-daemon-hxs7m\" (UID: \"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1\") " pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:23.224372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aa614955-d70e-4ab5-b9fb-6a58f218f477-kubelet-config\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:23.224744 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224392    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-cni-binary-copy\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.224744 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224423    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-sys\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.224744 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-multus-socket-dir-parent\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.224744 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224490    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-var-lib-cni-multus\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.224744 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224542    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-lib-modules\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.224744 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-var-lib-kubelet\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.224744 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-systemd\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.224744 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224618    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-var-lib-kubelet\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.224744 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-tuned\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.224744 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224676    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gz6tv"
Sep 11 13:21:23.225434 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225309    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 13:21:23.225434 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.224667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.225618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225448    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzxtz\" (UniqueName: \"kubernetes.io/projected/c236e5de-3078-4d4e-a4bf-588670685fa3-kube-api-access-mzxtz\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.225618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.225618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225520    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjtv6\" (UniqueName: \"kubernetes.io/projected/90842e4f-ef4d-490b-84f1-265e5b4d076f-kube-api-access-tjtv6\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.225618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225533    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zdc6h\""
Sep 11 13:21:23.225618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-sysctl-conf\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.225618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225584    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90842e4f-ef4d-490b-84f1-265e5b4d076f-cni-binary-copy\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.225618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-host\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.226011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225627    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-sys-fs\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.226011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225648    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-etc-selinux\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.226011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.226011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225687    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 13:21:23.226011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225719    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-modprobe-d\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.226011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225779    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 13:21:23.226011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c06f0042-4474-4b60-81a9-da7a8cb35ee8-tmp\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.226011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225828    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8bn4\" (UniqueName: \"kubernetes.io/projected/030910bd-d1c8-48c5-bc44-5e561b3fbf60-kube-api-access-f8bn4\") pod \"node-resolver-rkrhj\" (UID: \"030910bd-d1c8-48c5-bc44-5e561b3fbf60\") " pod="openshift-dns/node-resolver-rkrhj"
Sep 11 13:21:23.226011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225861    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-multus-cni-dir\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.226011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-run-netns\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.226011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225953    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 13:21:23.226011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.225994    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 13:21:23.226011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.226001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-var-lib-cni-bin\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.226583 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.226021    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 13:21:23.226583 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.226028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-run-multus-certs\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.226583 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.226055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aa614955-d70e-4ab5-b9fb-6a58f218f477-dbus\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:23.226583 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.226094    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-registration-dir\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.226583 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.226135    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/000a37ec-f62c-4808-a1ec-2992d0f919a4-agent-certs\") pod \"konnectivity-agent-xbv6v\" (UID: \"000a37ec-f62c-4808-a1ec-2992d0f919a4\") " pod="kube-system/konnectivity-agent-xbv6v"
Sep 11 13:21:23.226583 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.226164    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-cnibin\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.226583 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.226202    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-run-k8s-cni-cncf-io\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.226583 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.226229    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p5v6\" (UniqueName: \"kubernetes.io/projected/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-kube-api-access-6p5v6\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.226583 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.226263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhzhw\" (UniqueName: \"kubernetes.io/projected/c06f0042-4474-4b60-81a9-da7a8cb35ee8-kube-api-access-fhzhw\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.226583 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.226286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-etc-kubernetes\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.226583 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.226315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs\") pod \"network-metrics-daemon-hxs7m\" (UID: \"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1\") " pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:23.226583 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.226338    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-kubernetes\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.226583 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.226359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-sysctl-d\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.231973 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.231719    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 13:21:23.231973 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.231740    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ptcgg\""
Sep 11 13:21:23.231973 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.231773    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 13:21:23.231973 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.231957    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 13:21:23.232259 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.232002    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-q5hwt\""
Sep 11 13:21:23.232259 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.232175    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 13:21:23.232727 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.232703    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 13:21:23.232923 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.232882    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 13:21:23.233053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.233016    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 13:21:23.233335 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.233314    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:21:23.233418 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.233340    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-58zbx\""
Sep 11 13:21:23.233418 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.233380    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 13:21:23.233418 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.233390    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 13:21:23.233570 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.233344    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gsww4\""
Sep 11 13:21:23.267373 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.267209    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:21:23.302391 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.302329    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7f6tv"
Sep 11 13:21:23.312990 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.312956    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7f6tv"
Sep 11 13:21:23.313565 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.313543    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 13:21:23.313668 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.313575    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-132.ec2.internal" event={"ID":"102eeadc7fecd8005cb07392235d7fc0","Type":"ContainerStarted","Data":"5287d0891bc597b8e330536f52a7f7eac67420647b2d3aff8278e215433b7427"}
Sep 11 13:21:23.314849 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.314826    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal" event={"ID":"a39b3bf3052335ff9b476d16bbcf3e94","Type":"ContainerStarted","Data":"b041dc8d6f79285a099536fa5f1bd62e518a4db83e7513198e83adc09c613bbb"}
Sep 11 13:21:23.327108 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327084    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-system-cni-dir\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.327219 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.327219 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-run\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.327219 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3b3d505-6502-4cc5-9aec-aef4cd996049-host\") pod \"node-ca-gz6tv\" (UID: \"c3b3d505-6502-4cc5-9aec-aef4cd996049\") " pod="openshift-image-registry/node-ca-gz6tv"
Sep 11 13:21:23.327219 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327216    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-system-cni-dir\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.327454 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327219    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-run\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.327454 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-run-ovn-kubernetes\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.327454 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327302    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmxx6\" (UniqueName: \"kubernetes.io/projected/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-kube-api-access-jmxx6\") pod \"network-metrics-daemon-hxs7m\" (UID: \"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1\") " pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:23.327454 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aa614955-d70e-4ab5-b9fb-6a58f218f477-kubelet-config\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:23.327454 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327355    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-cni-binary-copy\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.327454 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-sys\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.327454 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-multus-socket-dir-parent\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.327454 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-var-lib-cni-multus\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.327454 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327432    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aa614955-d70e-4ab5-b9fb-6a58f218f477-kubelet-config\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327456    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swbh5\" (UniqueName: \"kubernetes.io/projected/2e81b8eb-767d-40c4-9d53-5d4c4428571d-kube-api-access-swbh5\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327483    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-sys\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-lib-modules\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-multus-socket-dir-parent\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-var-lib-kubelet\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkrlz\" (UniqueName: \"kubernetes.io/projected/c3b3d505-6502-4cc5-9aec-aef4cd996049-kube-api-access-tkrlz\") pod \"node-ca-gz6tv\" (UID: \"c3b3d505-6502-4cc5-9aec-aef4cd996049\") " pod="openshift-image-registry/node-ca-gz6tv"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327602    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8x7w\" (UniqueName: \"kubernetes.io/projected/d5446d61-4862-41c1-896d-49ad2cc64503-kube-api-access-v8x7w\") pod \"iptables-alerter-46b7m\" (UID: \"d5446d61-4862-41c1-896d-49ad2cc64503\") " pod="openshift-network-operator/iptables-alerter-46b7m"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327627    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2e81b8eb-767d-40c4-9d53-5d4c4428571d-env-overrides\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-systemd\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-var-lib-kubelet\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327700    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-var-lib-kubelet\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327716    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-tuned\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-lib-modules\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327762    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-systemd\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.327826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327819    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-var-lib-cni-multus\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-var-lib-kubelet\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzxtz\" (UniqueName: \"kubernetes.io/projected/c236e5de-3078-4d4e-a4bf-588670685fa3-kube-api-access-mzxtz\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327886    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-run-netns\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327962    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-cni-binary-copy\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.327985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjtv6\" (UniqueName: \"kubernetes.io/projected/90842e4f-ef4d-490b-84f1-265e5b4d076f-kube-api-access-tjtv6\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328013    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5446d61-4862-41c1-896d-49ad2cc64503-host-slash\") pod \"iptables-alerter-46b7m\" (UID: \"d5446d61-4862-41c1-896d-49ad2cc64503\") " pod="openshift-network-operator/iptables-alerter-46b7m"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328039    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-sysctl-conf\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90842e4f-ef4d-490b-84f1-265e5b4d076f-cni-binary-copy\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d5446d61-4862-41c1-896d-49ad2cc64503-iptables-alerter-script\") pod \"iptables-alerter-46b7m\" (UID: \"d5446d61-4862-41c1-896d-49ad2cc64503\") " pod="openshift-network-operator/iptables-alerter-46b7m"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-cni-bin\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328138    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-cni-netd\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-host\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-sys-fs\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328208    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-etc-selinux\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.328427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328230    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-sysctl-conf\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3b3d505-6502-4cc5-9aec-aef4cd996049-serviceca\") pod \"node-ca-gz6tv\" (UID: \"c3b3d505-6502-4cc5-9aec-aef4cd996049\") " pod="openshift-image-registry/node-ca-gz6tv"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-var-lib-openvswitch\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-log-socket\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328328    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-modprobe-d\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328367    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328370    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-host\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328423    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-etc-selinux\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328424    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c06f0042-4474-4b60-81a9-da7a8cb35ee8-tmp\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-sys-fs\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328462    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8bn4\" (UniqueName: \"kubernetes.io/projected/030910bd-d1c8-48c5-bc44-5e561b3fbf60-kube-api-access-f8bn4\") pod \"node-resolver-rkrhj\" (UID: \"030910bd-d1c8-48c5-bc44-5e561b3fbf60\") " pod="openshift-dns/node-resolver-rkrhj"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-multus-cni-dir\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328508    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-run-netns\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328529    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-modprobe-d\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328532    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-var-lib-cni-bin\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.329053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328577    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-var-lib-cni-bin\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328584    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-multus-cni-dir\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-run-multus-certs\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328618    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-run-netns\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aa614955-d70e-4ab5-b9fb-6a58f218f477-dbus\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328650    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90842e4f-ef4d-490b-84f1-265e5b4d076f-cni-binary-copy\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328658    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-run-multus-certs\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-registration-dir\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/000a37ec-f62c-4808-a1ec-2992d0f919a4-agent-certs\") pod \"konnectivity-agent-xbv6v\" (UID: \"000a37ec-f62c-4808-a1ec-2992d0f919a4\") " pod="kube-system/konnectivity-agent-xbv6v"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328708    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aa614955-d70e-4ab5-b9fb-6a58f218f477-dbus\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-cnibin\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328739    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-run-k8s-cni-cncf-io\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328752    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-registration-dir\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p5v6\" (UniqueName: \"kubernetes.io/projected/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-kube-api-access-6p5v6\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-cnibin\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-run-systemd\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328839    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-etc-openvswitch\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhzhw\" (UniqueName: \"kubernetes.io/projected/c06f0042-4474-4b60-81a9-da7a8cb35ee8-kube-api-access-fhzhw\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.329781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-etc-kubernetes\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328787    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-host-run-k8s-cni-cncf-io\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328931    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs\") pod \"network-metrics-daemon-hxs7m\" (UID: \"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1\") " pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.328963    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skbss\" (UniqueName: \"kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss\") pod \"network-check-target-2d9t8\" (UID: \"7f6d1bd5-a91f-4e7e-a071-f71eb67dc651\") " pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329019    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-kubelet\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329037    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-etc-kubernetes\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329068    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2e81b8eb-767d-40c4-9d53-5d4c4428571d-ovnkube-script-lib\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:23.329121    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-kubernetes\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-sysctl-d\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-socket-dir\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329218    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-kubernetes\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:23.329284    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs podName:30dcd97e-d15b-4ca5-ac15-997c0ba69cf1 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:23.829247867 +0000 UTC m=+3.047785354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs") pod "network-metrics-daemon-hxs7m" (UID: "30dcd97e-d15b-4ca5-ac15-997c0ba69cf1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329305    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-socket-dir\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329309    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-sysctl-d\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-system-cni-dir\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329356    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-slash\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.330636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329361    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-system-cni-dir\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329376    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e81b8eb-767d-40c4-9d53-5d4c4428571d-ovn-node-metrics-cert\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-multus-conf-dir\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90842e4f-ef4d-490b-84f1-265e5b4d076f-multus-daemon-config\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-run-ovn\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329461    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-multus-conf-dir\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/000a37ec-f62c-4808-a1ec-2992d0f919a4-konnectivity-ca\") pod \"konnectivity-agent-xbv6v\" (UID: \"000a37ec-f62c-4808-a1ec-2992d0f919a4\") " pod="kube-system/konnectivity-agent-xbv6v"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329494    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-hostroot\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-cnibin\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329553    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-run-openvswitch\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329601    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-device-dir\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/030910bd-d1c8-48c5-bc44-5e561b3fbf60-hosts-file\") pod \"node-resolver-rkrhj\" (UID: \"030910bd-d1c8-48c5-bc44-5e561b3fbf60\") " pod="openshift-dns/node-resolver-rkrhj"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329640    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2e81b8eb-767d-40c4-9d53-5d4c4428571d-ovnkube-config\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-sysconfig\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/030910bd-d1c8-48c5-bc44-5e561b3fbf60-tmp-dir\") pod \"node-resolver-rkrhj\" (UID: \"030910bd-d1c8-48c5-bc44-5e561b3fbf60\") " pod="openshift-dns/node-resolver-rkrhj"
Sep 11 13:21:23.331502 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-systemd-units\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.332320 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329716    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-node-log\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.332320 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/90842e4f-ef4d-490b-84f1-265e5b4d076f-hostroot\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.332320 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/90842e4f-ef4d-490b-84f1-265e5b4d076f-multus-daemon-config\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.332320 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329893    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/030910bd-d1c8-48c5-bc44-5e561b3fbf60-hosts-file\") pod \"node-resolver-rkrhj\" (UID: \"030910bd-d1c8-48c5-bc44-5e561b3fbf60\") " pod="openshift-dns/node-resolver-rkrhj"
Sep 11 13:21:23.332320 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:23.329880    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:23.332320 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329970    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-sysconfig\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.332320 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.329990    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/000a37ec-f62c-4808-a1ec-2992d0f919a4-konnectivity-ca\") pod \"konnectivity-agent-xbv6v\" (UID: \"000a37ec-f62c-4808-a1ec-2992d0f919a4\") " pod="kube-system/konnectivity-agent-xbv6v"
Sep 11 13:21:23.332320 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.330008    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-cnibin\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.332320 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:23.329975    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret podName:aa614955-d70e-4ab5-b9fb-6a58f218f477 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:23.829961715 +0000 UTC m=+3.048499204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret") pod "global-pull-secret-syncer-fld56" (UID: "aa614955-d70e-4ab5-b9fb-6a58f218f477") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:23.332320 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.330042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c236e5de-3078-4d4e-a4bf-588670685fa3-device-dir\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.332320 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.330295    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/030910bd-d1c8-48c5-bc44-5e561b3fbf60-tmp-dir\") pod \"node-resolver-rkrhj\" (UID: \"030910bd-d1c8-48c5-bc44-5e561b3fbf60\") " pod="openshift-dns/node-resolver-rkrhj"
Sep 11 13:21:23.332848 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.332581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c06f0042-4474-4b60-81a9-da7a8cb35ee8-tmp\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.332848 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.332619    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c06f0042-4474-4b60-81a9-da7a8cb35ee8-etc-tuned\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.333927 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.333693    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/000a37ec-f62c-4808-a1ec-2992d0f919a4-agent-certs\") pod \"konnectivity-agent-xbv6v\" (UID: \"000a37ec-f62c-4808-a1ec-2992d0f919a4\") " pod="kube-system/konnectivity-agent-xbv6v"
Sep 11 13:21:23.357444 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.357407    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p5v6\" (UniqueName: \"kubernetes.io/projected/9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab-kube-api-access-6p5v6\") pod \"multus-additional-cni-plugins-7lw4p\" (UID: \"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab\") " pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.357444 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.357427    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjtv6\" (UniqueName: \"kubernetes.io/projected/90842e4f-ef4d-490b-84f1-265e5b4d076f-kube-api-access-tjtv6\") pod \"multus-l8fqj\" (UID: \"90842e4f-ef4d-490b-84f1-265e5b4d076f\") " pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.357618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.357418    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8bn4\" (UniqueName: \"kubernetes.io/projected/030910bd-d1c8-48c5-bc44-5e561b3fbf60-kube-api-access-f8bn4\") pod \"node-resolver-rkrhj\" (UID: \"030910bd-d1c8-48c5-bc44-5e561b3fbf60\") " pod="openshift-dns/node-resolver-rkrhj"
Sep 11 13:21:23.357618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.357409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhzhw\" (UniqueName: \"kubernetes.io/projected/c06f0042-4474-4b60-81a9-da7a8cb35ee8-kube-api-access-fhzhw\") pod \"tuned-pqdpt\" (UID: \"c06f0042-4474-4b60-81a9-da7a8cb35ee8\") " pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.358560 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.358534    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmxx6\" (UniqueName: \"kubernetes.io/projected/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-kube-api-access-jmxx6\") pod \"network-metrics-daemon-hxs7m\" (UID: \"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1\") " pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:23.371850 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.371829    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzxtz\" (UniqueName: \"kubernetes.io/projected/c236e5de-3078-4d4e-a4bf-588670685fa3-kube-api-access-mzxtz\") pod \"aws-ebs-csi-driver-node-slvhg\" (UID: \"c236e5de-3078-4d4e-a4bf-588670685fa3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.430707 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430676    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swbh5\" (UniqueName: \"kubernetes.io/projected/2e81b8eb-767d-40c4-9d53-5d4c4428571d-kube-api-access-swbh5\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.430707 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkrlz\" (UniqueName: \"kubernetes.io/projected/c3b3d505-6502-4cc5-9aec-aef4cd996049-kube-api-access-tkrlz\") pod \"node-ca-gz6tv\" (UID: \"c3b3d505-6502-4cc5-9aec-aef4cd996049\") " pod="openshift-image-registry/node-ca-gz6tv"
Sep 11 13:21:23.430991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8x7w\" (UniqueName: \"kubernetes.io/projected/d5446d61-4862-41c1-896d-49ad2cc64503-kube-api-access-v8x7w\") pod \"iptables-alerter-46b7m\" (UID: \"d5446d61-4862-41c1-896d-49ad2cc64503\") " pod="openshift-network-operator/iptables-alerter-46b7m"
Sep 11 13:21:23.430991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430752    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2e81b8eb-767d-40c4-9d53-5d4c4428571d-env-overrides\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.430991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-run-netns\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.430991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5446d61-4862-41c1-896d-49ad2cc64503-host-slash\") pod \"iptables-alerter-46b7m\" (UID: \"d5446d61-4862-41c1-896d-49ad2cc64503\") " pod="openshift-network-operator/iptables-alerter-46b7m"
Sep 11 13:21:23.430991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430828    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d5446d61-4862-41c1-896d-49ad2cc64503-iptables-alerter-script\") pod \"iptables-alerter-46b7m\" (UID: \"d5446d61-4862-41c1-896d-49ad2cc64503\") " pod="openshift-network-operator/iptables-alerter-46b7m"
Sep 11 13:21:23.430991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-cni-bin\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.430991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-run-netns\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.430991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430875    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-cni-netd\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.430991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430929    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3b3d505-6502-4cc5-9aec-aef4cd996049-serviceca\") pod \"node-ca-gz6tv\" (UID: \"c3b3d505-6502-4cc5-9aec-aef4cd996049\") " pod="openshift-image-registry/node-ca-gz6tv"
Sep 11 13:21:23.430991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430940    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-cni-netd\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.430991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430950    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5446d61-4862-41c1-896d-49ad2cc64503-host-slash\") pod \"iptables-alerter-46b7m\" (UID: \"d5446d61-4862-41c1-896d-49ad2cc64503\") " pod="openshift-network-operator/iptables-alerter-46b7m"
Sep 11 13:21:23.430991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430956    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-var-lib-openvswitch\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.430991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430982    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-cni-bin\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.430991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.430987    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-log-socket\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431019    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-var-lib-openvswitch\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-run-systemd\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-etc-openvswitch\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431078    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-run-systemd\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431053    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-log-socket\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skbss\" (UniqueName: \"kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss\") pod \"network-check-target-2d9t8\" (UID: \"7f6d1bd5-a91f-4e7e-a071-f71eb67dc651\") " pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431110    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-etc-openvswitch\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-kubelet\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2e81b8eb-767d-40c4-9d53-5d4c4428571d-ovnkube-script-lib\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-slash\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431181    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-kubelet\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e81b8eb-767d-40c4-9d53-5d4c4428571d-ovn-node-metrics-cert\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-run-ovn\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-run-openvswitch\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431305    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2e81b8eb-767d-40c4-9d53-5d4c4428571d-env-overrides\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.431521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431356    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431398    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3b3d505-6502-4cc5-9aec-aef4cd996049-serviceca\") pod \"node-ca-gz6tv\" (UID: \"c3b3d505-6502-4cc5-9aec-aef4cd996049\") " pod="openshift-image-registry/node-ca-gz6tv"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431401    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-slash\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431431    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-run-openvswitch\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2e81b8eb-767d-40c4-9d53-5d4c4428571d-ovnkube-config\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431462    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-run-ovn\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-systemd-units\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431499    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-systemd-units\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-node-log\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431518    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d5446d61-4862-41c1-896d-49ad2cc64503-iptables-alerter-script\") pod \"iptables-alerter-46b7m\" (UID: \"d5446d61-4862-41c1-896d-49ad2cc64503\") " pod="openshift-network-operator/iptables-alerter-46b7m"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3b3d505-6502-4cc5-9aec-aef4cd996049-host\") pod \"node-ca-gz6tv\" (UID: \"c3b3d505-6502-4cc5-9aec-aef4cd996049\") " pod="openshift-image-registry/node-ca-gz6tv"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-node-log\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431556    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-run-ovn-kubernetes\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431568    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3b3d505-6502-4cc5-9aec-aef4cd996049-host\") pod \"node-ca-gz6tv\" (UID: \"c3b3d505-6502-4cc5-9aec-aef4cd996049\") " pod="openshift-image-registry/node-ca-gz6tv"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431578    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2e81b8eb-767d-40c4-9d53-5d4c4428571d-host-run-ovn-kubernetes\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.431741    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2e81b8eb-767d-40c4-9d53-5d4c4428571d-ovnkube-script-lib\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.432331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.432021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2e81b8eb-767d-40c4-9d53-5d4c4428571d-ovnkube-config\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.433605 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.433586    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2e81b8eb-767d-40c4-9d53-5d4c4428571d-ovn-node-metrics-cert\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.450708 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.450688    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkrlz\" (UniqueName: \"kubernetes.io/projected/c3b3d505-6502-4cc5-9aec-aef4cd996049-kube-api-access-tkrlz\") pod \"node-ca-gz6tv\" (UID: \"c3b3d505-6502-4cc5-9aec-aef4cd996049\") " pod="openshift-image-registry/node-ca-gz6tv"
Sep 11 13:21:23.455570 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:23.455548    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:21:23.455675 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:23.455574    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:21:23.455675 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:23.455589    2580 projected.go:196] Error preparing data for projected volume kube-api-access-skbss for pod openshift-network-diagnostics/network-check-target-2d9t8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:23.455675 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:23.455657    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss podName:7f6d1bd5-a91f-4e7e-a071-f71eb67dc651 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:23.955638308 +0000 UTC m=+3.174175794 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-skbss" (UniqueName: "kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss") pod "network-check-target-2d9t8" (UID: "7f6d1bd5-a91f-4e7e-a071-f71eb67dc651") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:23.470743 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.470723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8x7w\" (UniqueName: \"kubernetes.io/projected/d5446d61-4862-41c1-896d-49ad2cc64503-kube-api-access-v8x7w\") pod \"iptables-alerter-46b7m\" (UID: \"d5446d61-4862-41c1-896d-49ad2cc64503\") " pod="openshift-network-operator/iptables-alerter-46b7m"
Sep 11 13:21:23.481867 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.481842    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swbh5\" (UniqueName: \"kubernetes.io/projected/2e81b8eb-767d-40c4-9d53-5d4c4428571d-kube-api-access-swbh5\") pod \"ovnkube-node-cl86h\" (UID: \"2e81b8eb-767d-40c4-9d53-5d4c4428571d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.513286 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.513253    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pqdpt"
Sep 11 13:21:23.524206 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.524154    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg"
Sep 11 13:21:23.533975 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.533955    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rkrhj"
Sep 11 13:21:23.538512 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.538496    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7lw4p"
Sep 11 13:21:23.546815 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.546796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xbv6v"
Sep 11 13:21:23.555501 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.555440    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l8fqj"
Sep 11 13:21:23.562116 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.562096    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:23.570281 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.570263    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-46b7m"
Sep 11 13:21:23.576433 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.576411    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gz6tv"
Sep 11 13:21:23.833960 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.833870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:23.834097 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:23.833971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs\") pod \"network-metrics-daemon-hxs7m\" (UID: \"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1\") " pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:23.834097 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:23.834032    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:23.834097 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:23.834060    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:23.834190 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:23.834120    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs podName:30dcd97e-d15b-4ca5-ac15-997c0ba69cf1 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:24.834096395 +0000 UTC m=+4.052633882 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs") pod "network-metrics-daemon-hxs7m" (UID: "30dcd97e-d15b-4ca5-ac15-997c0ba69cf1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:23.834190 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:23.834137    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret podName:aa614955-d70e-4ab5-b9fb-6a58f218f477 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:24.834128487 +0000 UTC m=+4.052665957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret") pod "global-pull-secret-syncer-fld56" (UID: "aa614955-d70e-4ab5-b9fb-6a58f218f477") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:24.035205 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:24.035161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skbss\" (UniqueName: \"kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss\") pod \"network-check-target-2d9t8\" (UID: \"7f6d1bd5-a91f-4e7e-a071-f71eb67dc651\") " pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:24.035394 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:24.035343    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:21:24.035394 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:24.035363    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:21:24.035394 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:24.035372    2580 projected.go:196] Error preparing data for projected volume kube-api-access-skbss for pod openshift-network-diagnostics/network-check-target-2d9t8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:24.035541 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:24.035426    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss podName:7f6d1bd5-a91f-4e7e-a071-f71eb67dc651 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:25.035408773 +0000 UTC m=+4.253946260 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-skbss" (UniqueName: "kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss") pod "network-check-target-2d9t8" (UID: "7f6d1bd5-a91f-4e7e-a071-f71eb67dc651") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:24.309161 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:24.308995    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:24.309535 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:24.309015    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:24.309535 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:24.309250    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:24.309535 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:24.309298    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:24.315496 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:24.315222    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3b3d505_6502_4cc5_9aec_aef4cd996049.slice/crio-327c1e40c6d8a722cd18e99885aa6fbb7d0d84307a107901a71c079e9177c93b WatchSource:0}: Error finding container 327c1e40c6d8a722cd18e99885aa6fbb7d0d84307a107901a71c079e9177c93b: Status 404 returned error can't find the container with id 327c1e40c6d8a722cd18e99885aa6fbb7d0d84307a107901a71c079e9177c93b
Sep 11 13:21:24.315723 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:24.315671    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 13:16:23 +0000 UTC" deadline="2028-06-21 06:40:54.74805454 +0000 UTC"
Sep 11 13:21:24.315813 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:24.315723    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15569h19m30.432335598s"
Sep 11 13:21:24.319356 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:24.319328    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc236e5de_3078_4d4e_a4bf_588670685fa3.slice/crio-2a84d57a5a7fce7aedf2b4e01ec44043fd2d663d0df2c370b39097eeedf7af3b WatchSource:0}: Error finding container 2a84d57a5a7fce7aedf2b4e01ec44043fd2d663d0df2c370b39097eeedf7af3b: Status 404 returned error can't find the container with id 2a84d57a5a7fce7aedf2b4e01ec44043fd2d663d0df2c370b39097eeedf7af3b
Sep 11 13:21:24.320056 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:24.320010    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e81b8eb_767d_40c4_9d53_5d4c4428571d.slice/crio-5deaa870dd005334691da4b810a3dba521147d67bdfe3b198d8d80223debd0fe WatchSource:0}: Error finding container 5deaa870dd005334691da4b810a3dba521147d67bdfe3b198d8d80223debd0fe: Status 404 returned error can't find the container with id 5deaa870dd005334691da4b810a3dba521147d67bdfe3b198d8d80223debd0fe
Sep 11 13:21:24.320992 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:24.320967    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod000a37ec_f62c_4808_a1ec_2992d0f919a4.slice/crio-e19770e5ecd814b1f4331bd01c4ffda558395b0caeea11baa9734d803204a6ba WatchSource:0}: Error finding container e19770e5ecd814b1f4331bd01c4ffda558395b0caeea11baa9734d803204a6ba: Status 404 returned error can't find the container with id e19770e5ecd814b1f4331bd01c4ffda558395b0caeea11baa9734d803204a6ba
Sep 11 13:21:24.322079 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:24.322024    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90842e4f_ef4d_490b_84f1_265e5b4d076f.slice/crio-adfbe5d10995eac7d150f66a33d4f5bcbfa2131eaed5cffd628e6108c5a31d64 WatchSource:0}: Error finding container adfbe5d10995eac7d150f66a33d4f5bcbfa2131eaed5cffd628e6108c5a31d64: Status 404 returned error can't find the container with id adfbe5d10995eac7d150f66a33d4f5bcbfa2131eaed5cffd628e6108c5a31d64
Sep 11 13:21:24.325396 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:24.325379    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc06f0042_4474_4b60_81a9_da7a8cb35ee8.slice/crio-3ec195a039f5641bc48ba6cf55366c85357fa54e4930ac0ab1458daf3a06517c WatchSource:0}: Error finding container 3ec195a039f5641bc48ba6cf55366c85357fa54e4930ac0ab1458daf3a06517c: Status 404 returned error can't find the container with id 3ec195a039f5641bc48ba6cf55366c85357fa54e4930ac0ab1458daf3a06517c
Sep 11 13:21:24.326264 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:24.326245    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5446d61_4862_41c1_896d_49ad2cc64503.slice/crio-962d4e9b2610762a121f34a5a7d6d7dd8988ce05e820e23bca774aec06d36c6a WatchSource:0}: Error finding container 962d4e9b2610762a121f34a5a7d6d7dd8988ce05e820e23bca774aec06d36c6a: Status 404 returned error can't find the container with id 962d4e9b2610762a121f34a5a7d6d7dd8988ce05e820e23bca774aec06d36c6a
Sep 11 13:21:24.327387 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:24.327368    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2039e7_5e30_4f3f_b6ba_b94d7c0639ab.slice/crio-1813851108ff2437650ede0c3418a0a68d8d04cef407482557f98bde24dd2feb WatchSource:0}: Error finding container 1813851108ff2437650ede0c3418a0a68d8d04cef407482557f98bde24dd2feb: Status 404 returned error can't find the container with id 1813851108ff2437650ede0c3418a0a68d8d04cef407482557f98bde24dd2feb
Sep 11 13:21:24.328401 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:24.328311    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod030910bd_d1c8_48c5_bc44_5e561b3fbf60.slice/crio-5d1b4918a8722d4d0a9bb8db7e9bc05f03cb816a5122524976499c9ee0a49c66 WatchSource:0}: Error finding container 5d1b4918a8722d4d0a9bb8db7e9bc05f03cb816a5122524976499c9ee0a49c66: Status 404 returned error can't find the container with id 5d1b4918a8722d4d0a9bb8db7e9bc05f03cb816a5122524976499c9ee0a49c66
Sep 11 13:21:24.842172 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:24.842075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs\") pod \"network-metrics-daemon-hxs7m\" (UID: \"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1\") " pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:24.842172 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:24.842145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:24.842385 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:24.842267    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:24.842385 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:24.842334    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret podName:aa614955-d70e-4ab5-b9fb-6a58f218f477 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:26.842314328 +0000 UTC m=+6.060851804 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret") pod "global-pull-secret-syncer-fld56" (UID: "aa614955-d70e-4ab5-b9fb-6a58f218f477") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:24.842800 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:24.842782    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:24.842881 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:24.842835    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs podName:30dcd97e-d15b-4ca5-ac15-997c0ba69cf1 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:26.842818556 +0000 UTC m=+6.061356032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs") pod "network-metrics-daemon-hxs7m" (UID: "30dcd97e-d15b-4ca5-ac15-997c0ba69cf1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:25.043284 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.043240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skbss\" (UniqueName: \"kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss\") pod \"network-check-target-2d9t8\" (UID: \"7f6d1bd5-a91f-4e7e-a071-f71eb67dc651\") " pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:25.043567 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:25.043546    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:21:25.043651 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:25.043579    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:21:25.043651 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:25.043594    2580 projected.go:196] Error preparing data for projected volume kube-api-access-skbss for pod openshift-network-diagnostics/network-check-target-2d9t8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:25.043754 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:25.043653    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss podName:7f6d1bd5-a91f-4e7e-a071-f71eb67dc651 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:27.043635178 +0000 UTC m=+6.262172664 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-skbss" (UniqueName: "kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss") pod "network-check-target-2d9t8" (UID: "7f6d1bd5-a91f-4e7e-a071-f71eb67dc651") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:25.312282 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.312207    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:25.313112 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:25.312341    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:25.316693 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.316660    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 13:16:23 +0000 UTC" deadline="2028-05-17 08:40:59.979159924 +0000 UTC"
Sep 11 13:21:25.316693 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.316692    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14731h19m34.662471901s"
Sep 11 13:21:25.322309 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.322277    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pqdpt" event={"ID":"c06f0042-4474-4b60-81a9-da7a8cb35ee8","Type":"ContainerStarted","Data":"3ec195a039f5641bc48ba6cf55366c85357fa54e4930ac0ab1458daf3a06517c"}
Sep 11 13:21:25.325557 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.325481    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gz6tv" event={"ID":"c3b3d505-6502-4cc5-9aec-aef4cd996049","Type":"ContainerStarted","Data":"327c1e40c6d8a722cd18e99885aa6fbb7d0d84307a107901a71c079e9177c93b"}
Sep 11 13:21:25.343433 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.343396    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-132.ec2.internal" event={"ID":"102eeadc7fecd8005cb07392235d7fc0","Type":"ContainerStarted","Data":"5f8ff9c37c1b5c9cf62373f106e60c85486ccb9f3bd3c7c4b6258bc8a9d1d850"}
Sep 11 13:21:25.350387 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.350353    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-46b7m" event={"ID":"d5446d61-4862-41c1-896d-49ad2cc64503","Type":"ContainerStarted","Data":"962d4e9b2610762a121f34a5a7d6d7dd8988ce05e820e23bca774aec06d36c6a"}
Sep 11 13:21:25.354890 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.354830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xbv6v" event={"ID":"000a37ec-f62c-4808-a1ec-2992d0f919a4","Type":"ContainerStarted","Data":"e19770e5ecd814b1f4331bd01c4ffda558395b0caeea11baa9734d803204a6ba"}
Sep 11 13:21:25.359236 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.359206    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h" event={"ID":"2e81b8eb-767d-40c4-9d53-5d4c4428571d","Type":"ContainerStarted","Data":"5deaa870dd005334691da4b810a3dba521147d67bdfe3b198d8d80223debd0fe"}
Sep 11 13:21:25.362757 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.362505    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-132.ec2.internal" podStartSLOduration=3.362486316 podStartE2EDuration="3.362486316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:25.361383653 +0000 UTC m=+4.579921144" watchObservedRunningTime="2026-09-11 13:21:25.362486316 +0000 UTC m=+4.581023809"
Sep 11 13:21:25.363607 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.363538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg" event={"ID":"c236e5de-3078-4d4e-a4bf-588670685fa3","Type":"ContainerStarted","Data":"2a84d57a5a7fce7aedf2b4e01ec44043fd2d663d0df2c370b39097eeedf7af3b"}
Sep 11 13:21:25.370598 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.370569    2580 generic.go:356] "Generic (PLEG): container finished" podID="a39b3bf3052335ff9b476d16bbcf3e94" containerID="2966d9dfd60a4a4dfa86a41dd9e046fb04b8ac4a1def8cc93b7fcc10448b8179" exitCode=0
Sep 11 13:21:25.370713 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.370646    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal" event={"ID":"a39b3bf3052335ff9b476d16bbcf3e94","Type":"ContainerDied","Data":"2966d9dfd60a4a4dfa86a41dd9e046fb04b8ac4a1def8cc93b7fcc10448b8179"}
Sep 11 13:21:25.396538 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.396467    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rkrhj" event={"ID":"030910bd-d1c8-48c5-bc44-5e561b3fbf60","Type":"ContainerStarted","Data":"5d1b4918a8722d4d0a9bb8db7e9bc05f03cb816a5122524976499c9ee0a49c66"}
Sep 11 13:21:25.412146 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.412110    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lw4p" event={"ID":"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab","Type":"ContainerStarted","Data":"1813851108ff2437650ede0c3418a0a68d8d04cef407482557f98bde24dd2feb"}
Sep 11 13:21:25.425757 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:25.425713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l8fqj" event={"ID":"90842e4f-ef4d-490b-84f1-265e5b4d076f","Type":"ContainerStarted","Data":"adfbe5d10995eac7d150f66a33d4f5bcbfa2131eaed5cffd628e6108c5a31d64"}
Sep 11 13:21:26.309565 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:26.309465    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:26.309732 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:26.309465    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:26.309732 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:26.309612    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:26.309732 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:26.309697    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:26.446057 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:26.445964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal" event={"ID":"a39b3bf3052335ff9b476d16bbcf3e94","Type":"ContainerStarted","Data":"2f3cc9428ae8362b100ecea05f8da34687c553e8b98ded0dfc5f4521f40d18d5"}
Sep 11 13:21:26.860530 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:26.860489    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs\") pod \"network-metrics-daemon-hxs7m\" (UID: \"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1\") " pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:26.860726 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:26.860562    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:26.860726 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:26.860706    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:26.860847 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:26.860772    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret podName:aa614955-d70e-4ab5-b9fb-6a58f218f477 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:30.86075327 +0000 UTC m=+10.079290746 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret") pod "global-pull-secret-syncer-fld56" (UID: "aa614955-d70e-4ab5-b9fb-6a58f218f477") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:26.860847 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:26.860843    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:26.861060 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:26.860877    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs podName:30dcd97e-d15b-4ca5-ac15-997c0ba69cf1 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:30.860865374 +0000 UTC m=+10.079402851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs") pod "network-metrics-daemon-hxs7m" (UID: "30dcd97e-d15b-4ca5-ac15-997c0ba69cf1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:27.062264 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:27.062226    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skbss\" (UniqueName: \"kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss\") pod \"network-check-target-2d9t8\" (UID: \"7f6d1bd5-a91f-4e7e-a071-f71eb67dc651\") " pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:27.062456 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:27.062423    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:21:27.062517 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:27.062477    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:21:27.062517 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:27.062491    2580 projected.go:196] Error preparing data for projected volume kube-api-access-skbss for pod openshift-network-diagnostics/network-check-target-2d9t8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:27.062612 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:27.062549    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss podName:7f6d1bd5-a91f-4e7e-a071-f71eb67dc651 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:31.062530734 +0000 UTC m=+10.281068209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-skbss" (UniqueName: "kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss") pod "network-check-target-2d9t8" (UID: "7f6d1bd5-a91f-4e7e-a071-f71eb67dc651") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:27.310046 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:27.309434    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:27.310046 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:27.309573    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:28.309998 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:28.309964    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:28.310447 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:28.310106    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:28.310511 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:28.310478    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:28.310607 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:28.310583    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:29.309214 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:29.309148    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:29.309398 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:29.309299    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:30.310144 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:30.309446    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:30.310144 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:30.309585    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:30.310144 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:30.309989    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:30.310144 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:30.310092    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:30.895918 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:30.895870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs\") pod \"network-metrics-daemon-hxs7m\" (UID: \"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1\") " pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:30.896113 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:30.895950    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:30.896113 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:30.896077    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:30.896278 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:30.896133    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:30.896278 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:30.896137    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret podName:aa614955-d70e-4ab5-b9fb-6a58f218f477 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:38.896118609 +0000 UTC m=+18.114656098 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret") pod "global-pull-secret-syncer-fld56" (UID: "aa614955-d70e-4ab5-b9fb-6a58f218f477") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:30.896278 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:30.896177    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs podName:30dcd97e-d15b-4ca5-ac15-997c0ba69cf1 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:38.896167011 +0000 UTC m=+18.114704496 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs") pod "network-metrics-daemon-hxs7m" (UID: "30dcd97e-d15b-4ca5-ac15-997c0ba69cf1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:31.097988 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:31.097922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skbss\" (UniqueName: \"kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss\") pod \"network-check-target-2d9t8\" (UID: \"7f6d1bd5-a91f-4e7e-a071-f71eb67dc651\") " pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:31.098180 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:31.098105    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:21:31.098180 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:31.098128    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:21:31.098180 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:31.098140    2580 projected.go:196] Error preparing data for projected volume kube-api-access-skbss for pod openshift-network-diagnostics/network-check-target-2d9t8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:31.098337 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:31.098201    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss podName:7f6d1bd5-a91f-4e7e-a071-f71eb67dc651 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:39.098183241 +0000 UTC m=+18.316720716 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-skbss" (UniqueName: "kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss") pod "network-check-target-2d9t8" (UID: "7f6d1bd5-a91f-4e7e-a071-f71eb67dc651") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:31.310072 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:31.309986    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:31.310234 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:31.310093    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:32.309305 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:32.309270    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:32.309488 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:32.309403    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:32.309488 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:32.309476    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:32.309606 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:32.309585    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:33.310140 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:33.309913    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:33.310584 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:33.310245    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:34.309825 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:34.309784    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:34.310022 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:34.309919    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:34.310022 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:34.309963    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:34.310122 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:34.310092    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:35.309312 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:35.309272    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:35.309833 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:35.309786    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:36.309451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:36.309411    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:36.309943 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:36.309479    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:36.309943 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:36.309619    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:36.309943 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:36.309749    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:37.309341 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:37.309307    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:37.309512 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:37.309436    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:38.309354 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:38.309317    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:38.309531 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:38.309322    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:38.309531 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:38.309418    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:38.309531 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:38.309499    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:38.954280 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:38.954243    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs\") pod \"network-metrics-daemon-hxs7m\" (UID: \"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1\") " pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:38.954468 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:38.954308    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:38.954468 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:38.954352    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:38.954468 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:38.954426    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs podName:30dcd97e-d15b-4ca5-ac15-997c0ba69cf1 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:54.954404649 +0000 UTC m=+34.172942124 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs") pod "network-metrics-daemon-hxs7m" (UID: "30dcd97e-d15b-4ca5-ac15-997c0ba69cf1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:38.954468 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:38.954426    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:38.954468 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:38.954460    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret podName:aa614955-d70e-4ab5-b9fb-6a58f218f477 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:54.95445067 +0000 UTC m=+34.172988140 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret") pod "global-pull-secret-syncer-fld56" (UID: "aa614955-d70e-4ab5-b9fb-6a58f218f477") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:39.156151 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:39.156111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skbss\" (UniqueName: \"kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss\") pod \"network-check-target-2d9t8\" (UID: \"7f6d1bd5-a91f-4e7e-a071-f71eb67dc651\") " pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:39.156326 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:39.156283    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:21:39.156326 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:39.156318    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:21:39.156445 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:39.156333    2580 projected.go:196] Error preparing data for projected volume kube-api-access-skbss for pod openshift-network-diagnostics/network-check-target-2d9t8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:39.156445 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:39.156402    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss podName:7f6d1bd5-a91f-4e7e-a071-f71eb67dc651 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:55.156381592 +0000 UTC m=+34.374919064 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-skbss" (UniqueName: "kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss") pod "network-check-target-2d9t8" (UID: "7f6d1bd5-a91f-4e7e-a071-f71eb67dc651") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:39.309676 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:39.309595    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:39.310138 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:39.309725    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:40.309855 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:40.309809    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:40.310364 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:40.309817    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:40.310364 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:40.309947    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:40.310364 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:40.310048    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:41.309685 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:41.309658    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:41.309862 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:41.309749    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:42.309803 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:42.309720    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:42.310200 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:42.309720    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:42.310200 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:42.309857    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:42.310200 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:42.309927    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:42.481435 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:42.481372    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rkrhj" event={"ID":"030910bd-d1c8-48c5-bc44-5e561b3fbf60","Type":"ContainerStarted","Data":"e87181537eeec782ef179baa80a5f1dfc8a908b3484d9d680e02f641949c8294"}
Sep 11 13:21:42.483283 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:42.483257    2580 generic.go:356] "Generic (PLEG): container finished" podID="9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab" containerID="9e32b4141694b2d6097f91359ae8090cc6ee90fffcf905af54ad43dee758c0e7" exitCode=0
Sep 11 13:21:42.483411 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:42.483341    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lw4p" event={"ID":"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab","Type":"ContainerDied","Data":"9e32b4141694b2d6097f91359ae8090cc6ee90fffcf905af54ad43dee758c0e7"}
Sep 11 13:21:42.485127 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:42.485091    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l8fqj" event={"ID":"90842e4f-ef4d-490b-84f1-265e5b4d076f","Type":"ContainerStarted","Data":"e6250b10830e73cfa4e0d6c32691560ce563403b7b3373d52dcb4028f6b1b341"}
Sep 11 13:21:42.487644 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:42.487619    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pqdpt" event={"ID":"c06f0042-4474-4b60-81a9-da7a8cb35ee8","Type":"ContainerStarted","Data":"dfdd5085c27f5bc523e45437f243e7add2c57ab2686eccabd7bbb879131308e6"}
Sep 11 13:21:42.489252 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:42.489232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gz6tv" event={"ID":"c3b3d505-6502-4cc5-9aec-aef4cd996049","Type":"ContainerStarted","Data":"974744c865d0ca617b1e505d23ed2cadfc9b5f483a7e3322f7b7c7acc7570d1b"}
Sep 11 13:21:42.490926 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:42.490892    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xbv6v" event={"ID":"000a37ec-f62c-4808-a1ec-2992d0f919a4","Type":"ContainerStarted","Data":"de213abad193683b88c56b695c287c64adc6a11fbec9c528a572625e3793f1e7"}
Sep 11 13:21:42.492461 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:42.492441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg" event={"ID":"c236e5de-3078-4d4e-a4bf-588670685fa3","Type":"ContainerStarted","Data":"a226e63d20b3f3377374402d04b94df1449f30792f0feaf3007dc05ea92aca89"}
Sep 11 13:21:42.515571 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:42.515525    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-132.ec2.internal" podStartSLOduration=20.515511053 podStartE2EDuration="20.515511053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:26.467898798 +0000 UTC m=+5.686436291" watchObservedRunningTime="2026-09-11 13:21:42.515511053 +0000 UTC m=+21.734048545"
Sep 11 13:21:42.555542 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:42.555495    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rkrhj" podStartSLOduration=4.076905547 podStartE2EDuration="21.555478532s" totalImagesPullingTime="17.478572985s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:42.515655815 +0000 UTC m=+21.734193299" watchObservedRunningTime="2026-09-11 13:21:42.555478532 +0000 UTC m=+21.774016023"
Sep 11 13:21:42.590668 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:42.590454    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xbv6v" podStartSLOduration=4.143514735 podStartE2EDuration="21.590435606s" totalImagesPullingTime="17.446920871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:42.590350285 +0000 UTC m=+21.808887778" watchObservedRunningTime="2026-09-11 13:21:42.590435606 +0000 UTC m=+21.808973099"
Sep 11 13:21:42.590799 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:42.590771    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pqdpt" podStartSLOduration=4.109280221 podStartE2EDuration="21.590759968s" totalImagesPullingTime="17.481479747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:42.555115557 +0000 UTC m=+21.773653051" watchObservedRunningTime="2026-09-11 13:21:42.590759968 +0000 UTC m=+21.809297472"
Sep 11 13:21:42.665027 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:42.664965    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gz6tv" podStartSLOduration=4.175319798 podStartE2EDuration="21.664937901s" totalImagesPullingTime="17.489618103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:42.622822415 +0000 UTC m=+21.841359906" watchObservedRunningTime="2026-09-11 13:21:42.664937901 +0000 UTC m=+21.883475399"
Sep 11 13:21:43.102590 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:43.102553    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 13:21:43.294188 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:43.294090    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T13:21:43.10257261Z","UUID":"4e56727d-5ccb-4826-b93f-79e6a68c226f","Handler":null,"Name":"","Endpoint":""}
Sep 11 13:21:43.295875 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:43.295847    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 13:21:43.296005 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:43.295884    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 13:21:43.309492 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:43.309463    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:43.309604 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:43.309579    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:43.497444 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:43.497394    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-46b7m" event={"ID":"d5446d61-4862-41c1-896d-49ad2cc64503","Type":"ContainerStarted","Data":"44c53a5a39a9fbf2726a7cddfc4e319cefe202f1ffad0c5c5215c5702480da99"}
Sep 11 13:21:43.501491 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:43.501455    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h" event={"ID":"2e81b8eb-767d-40c4-9d53-5d4c4428571d","Type":"ContainerStarted","Data":"ef73569b6b8ada39d4f02485d6d07cc4a015b24601a646f9b7bc8b30f2fb7b39"}
Sep 11 13:21:43.501619 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:43.501547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h" event={"ID":"2e81b8eb-767d-40c4-9d53-5d4c4428571d","Type":"ContainerStarted","Data":"8ab3d999169dd2866a864ef259c858f0b682ddff256d0e770b16eccdcd3a4a90"}
Sep 11 13:21:43.501619 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:43.501597    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h" event={"ID":"2e81b8eb-767d-40c4-9d53-5d4c4428571d","Type":"ContainerStarted","Data":"2ce99d4fabecf03e0dcf2eec6d6410c4ab86c5f26f2dfc2bf20f6bb0489991ff"}
Sep 11 13:21:43.501721 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:43.501645    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h" event={"ID":"2e81b8eb-767d-40c4-9d53-5d4c4428571d","Type":"ContainerStarted","Data":"1a222db0c43e82e6c369dde19dffbbc3b83bbe3dc087654cb8ab64ea8d257051"}
Sep 11 13:21:43.501721 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:43.501692    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h" event={"ID":"2e81b8eb-767d-40c4-9d53-5d4c4428571d","Type":"ContainerStarted","Data":"29607233a243260cdecea015b3cab749ac2fab4c8d96fdd5e359eaa946225a4c"}
Sep 11 13:21:43.501830 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:43.501726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h" event={"ID":"2e81b8eb-767d-40c4-9d53-5d4c4428571d","Type":"ContainerStarted","Data":"a7d5a8f2ffbe422a47c56b728a9510919b9fea3a2b0e06022dd43731a4735d1e"}
Sep 11 13:21:43.503572 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:43.503545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg" event={"ID":"c236e5de-3078-4d4e-a4bf-588670685fa3","Type":"ContainerStarted","Data":"765463de94bb15b9a33cbf6a87cb33a91dbcd03675ecbfb97bd79dbe880184bc"}
Sep 11 13:21:43.531237 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:43.531190    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-l8fqj" podStartSLOduration=4.894778183 podStartE2EDuration="22.531168521s" totalImagesPullingTime="17.636390338s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:42.715326149 +0000 UTC m=+21.933863664" watchObservedRunningTime="2026-09-11 13:21:43.531168521 +0000 UTC m=+22.749705991"
Sep 11 13:21:43.531809 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:43.531779    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-46b7m" podStartSLOduration=5.091028322 podStartE2EDuration="22.531766938s" totalImagesPullingTime="17.440738616s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:43.531283654 +0000 UTC m=+22.749821147" watchObservedRunningTime="2026-09-11 13:21:43.531766938 +0000 UTC m=+22.750304432"
Sep 11 13:21:44.309440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:44.309362    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:44.309660 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:44.309496    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:44.309660 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:44.309590    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:44.309744 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:44.309675    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:44.496848 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:44.496808    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xbv6v"
Sep 11 13:21:44.508874 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:44.508836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg" event={"ID":"c236e5de-3078-4d4e-a4bf-588670685fa3","Type":"ContainerStarted","Data":"5730da01e9178d30a425ffdabe98427971c07aa4f95bda99931c5408add1d947"}
Sep 11 13:21:45.316696 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:45.316669    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:45.317004 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:45.316783    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:45.436459 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:45.436404    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xbv6v"
Sep 11 13:21:45.437030 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:45.437004    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xbv6v"
Sep 11 13:21:45.458242 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:45.458182    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slvhg" podStartSLOduration=5.061675753 podStartE2EDuration="24.458166027s" totalImagesPullingTime="19.396490274s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:44.538733278 +0000 UTC m=+23.757270772" watchObservedRunningTime="2026-09-11 13:21:45.458166027 +0000 UTC m=+24.676703519"
Sep 11 13:21:45.543758 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:45.543731    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xbv6v"
Sep 11 13:21:46.309191 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:46.309155    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:46.309380 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:46.309167    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:46.309380 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:46.309273    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:46.309492 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:46.309378    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:46.517462 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:46.517422    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h" event={"ID":"2e81b8eb-767d-40c4-9d53-5d4c4428571d","Type":"ContainerStarted","Data":"c62d711d3ed605df9e5d0d842fe0b6b31b463a26c62787e6ab649fc4d953c1e2"}
Sep 11 13:21:47.309310 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:47.309278    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:47.309829 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:47.309413    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:48.309538 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:48.309351    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:48.310174 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:48.309350    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:48.310174 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:48.309625    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:48.310174 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:48.309679    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:48.525497 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:48.525454    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h" event={"ID":"2e81b8eb-767d-40c4-9d53-5d4c4428571d","Type":"ContainerStarted","Data":"55df8bf0be7b3ebf0396ccb878a8ebeeb48192cdf147b24fc0dfddc4498fdeb9"}
Sep 11 13:21:48.525725 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:48.525705    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:48.525826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:48.525784    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:48.530271 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:48.530244    2580 generic.go:356] "Generic (PLEG): container finished" podID="9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab" containerID="352bc2b26285053f8e9aef1e2cf39371981c67f535ff51745d451f27cdacf228" exitCode=0
Sep 11 13:21:48.530387 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:48.530293    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lw4p" event={"ID":"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab","Type":"ContainerDied","Data":"352bc2b26285053f8e9aef1e2cf39371981c67f535ff51745d451f27cdacf228"}
Sep 11 13:21:48.542604 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:48.542582    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:48.562429 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:48.562384    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h" podStartSLOduration=9.342407435 podStartE2EDuration="27.56237049s" totalImagesPullingTime="18.219963055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:48.560760094 +0000 UTC m=+27.779297585" watchObservedRunningTime="2026-09-11 13:21:48.56237049 +0000 UTC m=+27.780907981"
Sep 11 13:21:49.309950 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:49.309916    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:49.310422 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:49.310059    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:49.533392 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:49.533365    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:49.548321 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:49.548281    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:21:49.813117 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:49.812928    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2d9t8"]
Sep 11 13:21:49.813251 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:49.813196    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:49.813300 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:49.813282    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:49.815967 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:49.815939    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hxs7m"]
Sep 11 13:21:49.816078 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:49.816068    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:49.816210 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:49.816184    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:49.817897 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:49.817287    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fld56"]
Sep 11 13:21:49.817897 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:49.817451    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:49.817897 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:49.817572    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:50.537338 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:50.537305    2580 generic.go:356] "Generic (PLEG): container finished" podID="9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab" containerID="14b06313505aaf3d5cecdbb56262f58882f421d6c4f1988b4cc62904568613c0" exitCode=0
Sep 11 13:21:50.537701 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:50.537393    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lw4p" event={"ID":"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab","Type":"ContainerDied","Data":"14b06313505aaf3d5cecdbb56262f58882f421d6c4f1988b4cc62904568613c0"}
Sep 11 13:21:51.310657 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:51.310630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:51.310868 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:51.310736    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:51.310868 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:51.310740    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:51.310868 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:51.310849    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:51.311038 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:51.310877    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:51.311038 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:51.310960    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:51.541444 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:51.541412    2580 generic.go:356] "Generic (PLEG): container finished" podID="9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab" containerID="504196f4da4c7476b72590d4530322e135ea5347f89d7e19c7d9ab0b4e6c1f36" exitCode=0
Sep 11 13:21:51.541810 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:51.541493    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lw4p" event={"ID":"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab","Type":"ContainerDied","Data":"504196f4da4c7476b72590d4530322e135ea5347f89d7e19c7d9ab0b4e6c1f36"}
Sep 11 13:21:53.309921 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:53.309870    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:53.310321 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:53.309870    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:53.310321 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:53.310022    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:53.310321 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:53.310033    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:53.310321 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:53.310114    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:53.310321 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:53.310207    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:54.966402 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:54.966360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs\") pod \"network-metrics-daemon-hxs7m\" (UID: \"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1\") " pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:54.966948 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:54.966507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:54.966948 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:54.966526    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:54.966948 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:54.966610    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs podName:30dcd97e-d15b-4ca5-ac15-997c0ba69cf1 nodeName:}" failed. No retries permitted until 2026-09-11 13:22:26.966594975 +0000 UTC m=+66.185132444 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs") pod "network-metrics-daemon-hxs7m" (UID: "30dcd97e-d15b-4ca5-ac15-997c0ba69cf1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:54.966948 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:54.966656    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:54.966948 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:54.966724    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret podName:aa614955-d70e-4ab5-b9fb-6a58f218f477 nodeName:}" failed. No retries permitted until 2026-09-11 13:22:26.966706969 +0000 UTC m=+66.185244452 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret") pod "global-pull-secret-syncer-fld56" (UID: "aa614955-d70e-4ab5-b9fb-6a58f218f477") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:55.167556 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.167514    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skbss\" (UniqueName: \"kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss\") pod \"network-check-target-2d9t8\" (UID: \"7f6d1bd5-a91f-4e7e-a071-f71eb67dc651\") " pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:55.167744 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:55.167701    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:21:55.167744 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:55.167722    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:21:55.167744 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:55.167734    2580 projected.go:196] Error preparing data for projected volume kube-api-access-skbss for pod openshift-network-diagnostics/network-check-target-2d9t8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:55.167871 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:55.167800    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss podName:7f6d1bd5-a91f-4e7e-a071-f71eb67dc651 nodeName:}" failed. No retries permitted until 2026-09-11 13:22:27.167780142 +0000 UTC m=+66.386317622 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-skbss" (UniqueName: "kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss") pod "network-check-target-2d9t8" (UID: "7f6d1bd5-a91f-4e7e-a071-f71eb67dc651") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:55.309213 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.309123    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:55.309377 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.309253    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:55.309377 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:55.309265    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2d9t8" podUID="7f6d1bd5-a91f-4e7e-a071-f71eb67dc651"
Sep 11 13:21:55.309377 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.309284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:55.309377 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:55.309342    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fld56" podUID="aa614955-d70e-4ab5-b9fb-6a58f218f477"
Sep 11 13:21:55.309593 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:55.309432    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hxs7m" podUID="30dcd97e-d15b-4ca5-ac15-997c0ba69cf1"
Sep 11 13:21:55.498702 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.498491    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-132.ec2.internal" event="NodeReady"
Sep 11 13:21:55.498881 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.498851    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 13:21:55.580550 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.580506    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9wxh8"]
Sep 11 13:21:55.593695 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.593666    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5zw59"]
Sep 11 13:21:55.593812 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.593792    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:21:55.600088 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.598145    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 13:21:55.600088 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.599863    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 13:21:55.600279 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.600263    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vqfbt\""
Sep 11 13:21:55.623980 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.623937    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5zw59"]
Sep 11 13:21:55.623980 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.623988    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9wxh8"]
Sep 11 13:21:55.624172 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.623996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5zw59"
Sep 11 13:21:55.627949 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.627897    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 13:21:55.628297 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.628227    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5ng6x\""
Sep 11 13:21:55.628297 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.628275    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 13:21:55.628934 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.628895    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 13:21:55.672206 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.672174    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/934019ff-93c3-464b-906f-39c185f6d5e4-config-volume\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:21:55.672206 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.672208    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h95bm\" (UniqueName: \"kubernetes.io/projected/934019ff-93c3-464b-906f-39c185f6d5e4-kube-api-access-h95bm\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:21:55.672397 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.672234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/934019ff-93c3-464b-906f-39c185f6d5e4-tmp-dir\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:21:55.672397 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.672296    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:21:55.773565 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.773530    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h95bm\" (UniqueName: \"kubernetes.io/projected/934019ff-93c3-464b-906f-39c185f6d5e4-kube-api-access-h95bm\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:21:55.773748 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.773582    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/934019ff-93c3-464b-906f-39c185f6d5e4-tmp-dir\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:21:55.773748 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.773613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:21:55.773748 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.773678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc1353d3-7bc6-45bd-9ecf-c325b4469fb8-cert\") pod \"ingress-canary-5zw59\" (UID: \"fc1353d3-7bc6-45bd-9ecf-c325b4469fb8\") " pod="openshift-ingress-canary/ingress-canary-5zw59"
Sep 11 13:21:55.773748 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.773707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhn2g\" (UniqueName: \"kubernetes.io/projected/fc1353d3-7bc6-45bd-9ecf-c325b4469fb8-kube-api-access-qhn2g\") pod \"ingress-canary-5zw59\" (UID: \"fc1353d3-7bc6-45bd-9ecf-c325b4469fb8\") " pod="openshift-ingress-canary/ingress-canary-5zw59"
Sep 11 13:21:55.773748 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.773745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/934019ff-93c3-464b-906f-39c185f6d5e4-config-volume\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:21:55.773984 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:55.773781    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:21:55.773984 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:55.773845    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls podName:934019ff-93c3-464b-906f-39c185f6d5e4 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:56.273828541 +0000 UTC m=+35.492366025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls") pod "dns-default-9wxh8" (UID: "934019ff-93c3-464b-906f-39c185f6d5e4") : secret "dns-default-metrics-tls" not found
Sep 11 13:21:55.774057 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.773978    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/934019ff-93c3-464b-906f-39c185f6d5e4-tmp-dir\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:21:55.774254 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.774236    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/934019ff-93c3-464b-906f-39c185f6d5e4-config-volume\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:21:55.790072 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.790043    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h95bm\" (UniqueName: \"kubernetes.io/projected/934019ff-93c3-464b-906f-39c185f6d5e4-kube-api-access-h95bm\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:21:55.874212 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.874147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc1353d3-7bc6-45bd-9ecf-c325b4469fb8-cert\") pod \"ingress-canary-5zw59\" (UID: \"fc1353d3-7bc6-45bd-9ecf-c325b4469fb8\") " pod="openshift-ingress-canary/ingress-canary-5zw59"
Sep 11 13:21:55.874212 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.874180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhn2g\" (UniqueName: \"kubernetes.io/projected/fc1353d3-7bc6-45bd-9ecf-c325b4469fb8-kube-api-access-qhn2g\") pod \"ingress-canary-5zw59\" (UID: \"fc1353d3-7bc6-45bd-9ecf-c325b4469fb8\") " pod="openshift-ingress-canary/ingress-canary-5zw59"
Sep 11 13:21:55.876705 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.876673    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc1353d3-7bc6-45bd-9ecf-c325b4469fb8-cert\") pod \"ingress-canary-5zw59\" (UID: \"fc1353d3-7bc6-45bd-9ecf-c325b4469fb8\") " pod="openshift-ingress-canary/ingress-canary-5zw59"
Sep 11 13:21:55.884781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.884757    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhn2g\" (UniqueName: \"kubernetes.io/projected/fc1353d3-7bc6-45bd-9ecf-c325b4469fb8-kube-api-access-qhn2g\") pod \"ingress-canary-5zw59\" (UID: \"fc1353d3-7bc6-45bd-9ecf-c325b4469fb8\") " pod="openshift-ingress-canary/ingress-canary-5zw59"
Sep 11 13:21:55.932639 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:55.932610    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5zw59"
Sep 11 13:21:56.244305 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:56.244231    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5zw59"]
Sep 11 13:21:56.247890 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:21:56.247860    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc1353d3_7bc6_45bd_9ecf_c325b4469fb8.slice/crio-994252c5d7ffd45c23574ed1cc87e024d41d56546d0eefa2e7410d3557e7604a WatchSource:0}: Error finding container 994252c5d7ffd45c23574ed1cc87e024d41d56546d0eefa2e7410d3557e7604a: Status 404 returned error can't find the container with id 994252c5d7ffd45c23574ed1cc87e024d41d56546d0eefa2e7410d3557e7604a
Sep 11 13:21:56.277701 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:56.277674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:21:56.277884 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:56.277861    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:21:56.277994 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:56.277981    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls podName:934019ff-93c3-464b-906f-39c185f6d5e4 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:57.277955215 +0000 UTC m=+36.496492703 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls") pod "dns-default-9wxh8" (UID: "934019ff-93c3-464b-906f-39c185f6d5e4") : secret "dns-default-metrics-tls" not found
Sep 11 13:21:56.555058 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:56.554967    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5zw59" event={"ID":"fc1353d3-7bc6-45bd-9ecf-c325b4469fb8","Type":"ContainerStarted","Data":"994252c5d7ffd45c23574ed1cc87e024d41d56546d0eefa2e7410d3557e7604a"}
Sep 11 13:21:56.557375 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:56.557349    2580 generic.go:356] "Generic (PLEG): container finished" podID="9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab" containerID="43d3782467c4df238a79bf4f9000b7e7ed9a0f78bfa21b0eb9f739b732b10f5c" exitCode=0
Sep 11 13:21:56.557481 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:56.557395    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lw4p" event={"ID":"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab","Type":"ContainerDied","Data":"43d3782467c4df238a79bf4f9000b7e7ed9a0f78bfa21b0eb9f739b732b10f5c"}
Sep 11 13:21:57.284499 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:57.284460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:21:57.284967 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:57.284622    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:21:57.284967 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:57.284692    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls podName:934019ff-93c3-464b-906f-39c185f6d5e4 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:59.284676623 +0000 UTC m=+38.503214114 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls") pod "dns-default-9wxh8" (UID: "934019ff-93c3-464b-906f-39c185f6d5e4") : secret "dns-default-metrics-tls" not found
Sep 11 13:21:57.309708 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:57.309676    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:21:57.309873 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:57.309676    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:21:57.310300 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:57.309680    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:21:57.315658 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:57.315635    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:21:57.315831 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:57.315808    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:21:57.316126 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:57.316105    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:21:57.316224 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:57.316142    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:21:57.316224 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:57.316182    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j2l67\""
Sep 11 13:21:57.316882 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:57.316861    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tkfcx\""
Sep 11 13:21:57.564044 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:57.564012    2580 generic.go:356] "Generic (PLEG): container finished" podID="9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab" containerID="f9a96b6c9ed1cf9621d97bd5c29998e42ad2d97464735b9c88738f0a4f0b4068" exitCode=0
Sep 11 13:21:57.564220 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:57.564092    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lw4p" event={"ID":"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab","Type":"ContainerDied","Data":"f9a96b6c9ed1cf9621d97bd5c29998e42ad2d97464735b9c88738f0a4f0b4068"}
Sep 11 13:21:58.569643 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:58.569605    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7lw4p" event={"ID":"9d2039e7-5e30-4f3f-b6ba-b94d7c0639ab","Type":"ContainerStarted","Data":"e3cd672b624113ea2b3fd7bd01c639cf03acc41143a5585f8861bd34dc87a6bd"}
Sep 11 13:21:58.609879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:58.609826    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7lw4p" podStartSLOduration=8.925945607 podStartE2EDuration="37.609811084s" totalImagesPullingTime="28.683865477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:58.604667976 +0000 UTC m=+37.823205464" watchObservedRunningTime="2026-09-11 13:21:58.609811084 +0000 UTC m=+37.828348566"
Sep 11 13:21:59.302739 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:59.302693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:21:59.302980 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:59.302862    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:21:59.302980 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:21:59.302971    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls podName:934019ff-93c3-464b-906f-39c185f6d5e4 nodeName:}" failed. No retries permitted until 2026-09-11 13:22:03.302951541 +0000 UTC m=+42.521489025 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls") pod "dns-default-9wxh8" (UID: "934019ff-93c3-464b-906f-39c185f6d5e4") : secret "dns-default-metrics-tls" not found
Sep 11 13:21:59.573264 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:59.573232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5zw59" event={"ID":"fc1353d3-7bc6-45bd-9ecf-c325b4469fb8","Type":"ContainerStarted","Data":"b418df97c2bab2d369941d030a5850fd3b42e9ba2497ff00b5cfd3177392d4c4"}
Sep 11 13:21:59.607739 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:21:59.607692    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5zw59" podStartSLOduration=2.143260341 podStartE2EDuration="4.607675029s" totalImagesPullingTime="2.464414688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:59.607670334 +0000 UTC m=+38.826207826" watchObservedRunningTime="2026-09-11 13:21:59.607675029 +0000 UTC m=+38.826212528"
Sep 11 13:22:03.332925 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:03.332865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:22:03.333400 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:22:03.333077    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:22:03.333400 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:22:03.333168    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls podName:934019ff-93c3-464b-906f-39c185f6d5e4 nodeName:}" failed. No retries permitted until 2026-09-11 13:22:11.333144819 +0000 UTC m=+50.551682307 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls") pod "dns-default-9wxh8" (UID: "934019ff-93c3-464b-906f-39c185f6d5e4") : secret "dns-default-metrics-tls" not found
Sep 11 13:22:11.390535 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:11.390491    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:22:11.391124 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:22:11.390663    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:22:11.391124 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:22:11.390764    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls podName:934019ff-93c3-464b-906f-39c185f6d5e4 nodeName:}" failed. No retries permitted until 2026-09-11 13:22:27.39074059 +0000 UTC m=+66.609278068 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls") pod "dns-default-9wxh8" (UID: "934019ff-93c3-464b-906f-39c185f6d5e4") : secret "dns-default-metrics-tls" not found
Sep 11 13:22:21.552164 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:21.552137    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cl86h"
Sep 11 13:22:27.005605 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.005558    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:22:27.006002 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.005643    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs\") pod \"network-metrics-daemon-hxs7m\" (UID: \"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1\") " pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:22:27.008602 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.008584    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:22:27.008657 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.008628    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:22:27.015974 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:22:27.015954    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 13:22:27.016023 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:22:27.016015    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs podName:30dcd97e-d15b-4ca5-ac15-997c0ba69cf1 nodeName:}" failed. No retries permitted until 2026-09-11 13:23:31.01599776 +0000 UTC m=+130.234535243 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs") pod "network-metrics-daemon-hxs7m" (UID: "30dcd97e-d15b-4ca5-ac15-997c0ba69cf1") : secret "metrics-daemon-secret" not found
Sep 11 13:22:27.019170 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.019149    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aa614955-d70e-4ab5-b9fb-6a58f218f477-original-pull-secret\") pod \"global-pull-secret-syncer-fld56\" (UID: \"aa614955-d70e-4ab5-b9fb-6a58f218f477\") " pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:22:27.023131 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.023110    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fld56"
Sep 11 13:22:27.207040 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.206998    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skbss\" (UniqueName: \"kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss\") pod \"network-check-target-2d9t8\" (UID: \"7f6d1bd5-a91f-4e7e-a071-f71eb67dc651\") " pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:22:27.210128 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.210110    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:22:27.220276 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.220257    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:22:27.230063 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.230032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skbss\" (UniqueName: \"kubernetes.io/projected/7f6d1bd5-a91f-4e7e-a071-f71eb67dc651-kube-api-access-skbss\") pod \"network-check-target-2d9t8\" (UID: \"7f6d1bd5-a91f-4e7e-a071-f71eb67dc651\") " pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:22:27.323162 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.323133    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fld56"]
Sep 11 13:22:27.326586 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:22:27.326554    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa614955_d70e_4ab5_b9fb_6a58f218f477.slice/crio-c777d3fc718d894e1514386601debc48a7d3a88d5e9f5f3ded94502332e460fc WatchSource:0}: Error finding container c777d3fc718d894e1514386601debc48a7d3a88d5e9f5f3ded94502332e460fc: Status 404 returned error can't find the container with id c777d3fc718d894e1514386601debc48a7d3a88d5e9f5f3ded94502332e460fc
Sep 11 13:22:27.332916 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.332890    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tkfcx\""
Sep 11 13:22:27.340794 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.340775    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:22:27.408598 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.408568    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:22:27.408750 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:22:27.408736    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:22:27.408831 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:22:27.408819    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls podName:934019ff-93c3-464b-906f-39c185f6d5e4 nodeName:}" failed. No retries permitted until 2026-09-11 13:22:59.408794234 +0000 UTC m=+98.627331717 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls") pod "dns-default-9wxh8" (UID: "934019ff-93c3-464b-906f-39c185f6d5e4") : secret "dns-default-metrics-tls" not found
Sep 11 13:22:27.608804 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.608719    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2d9t8"]
Sep 11 13:22:27.611328 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:22:27.611300    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f6d1bd5_a91f_4e7e_a071_f71eb67dc651.slice/crio-2e1b3d007e88c679f92ac86e1b23afe27b412a3dad024f38a412bdda7cb375ef WatchSource:0}: Error finding container 2e1b3d007e88c679f92ac86e1b23afe27b412a3dad024f38a412bdda7cb375ef: Status 404 returned error can't find the container with id 2e1b3d007e88c679f92ac86e1b23afe27b412a3dad024f38a412bdda7cb375ef
Sep 11 13:22:27.639141 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.639108    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fld56" event={"ID":"aa614955-d70e-4ab5-b9fb-6a58f218f477","Type":"ContainerStarted","Data":"c777d3fc718d894e1514386601debc48a7d3a88d5e9f5f3ded94502332e460fc"}
Sep 11 13:22:27.640156 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:27.640133    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2d9t8" event={"ID":"7f6d1bd5-a91f-4e7e-a071-f71eb67dc651","Type":"ContainerStarted","Data":"2e1b3d007e88c679f92ac86e1b23afe27b412a3dad024f38a412bdda7cb375ef"}
Sep 11 13:22:32.655180 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:32.655143    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2d9t8" event={"ID":"7f6d1bd5-a91f-4e7e-a071-f71eb67dc651","Type":"ContainerStarted","Data":"cb8cd0524033fb772ad7c307bdbe2ce12f1b2ff0a75c4174c752002c2a67d175"}
Sep 11 13:22:32.655598 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:32.655226    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:22:32.656596 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:32.656575    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fld56" event={"ID":"aa614955-d70e-4ab5-b9fb-6a58f218f477","Type":"ContainerStarted","Data":"d69bf25fb2de969ff5a27b03c855be7689503791a497851dec57d9f4b27108ab"}
Sep 11 13:22:32.689790 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:32.689715    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2d9t8" podStartSLOduration=67.495607563 podStartE2EDuration="1m11.689694173s" totalImagesPullingTime="4.19408661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:32.688165398 +0000 UTC m=+71.906702894" watchObservedRunningTime="2026-09-11 13:22:32.689694173 +0000 UTC m=+71.908231667"
Sep 11 13:22:32.724449 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:32.724393    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fld56" podStartSLOduration=67.241821686 podStartE2EDuration="1m11.7243764s" totalImagesPullingTime="4.482554714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:32.72169453 +0000 UTC m=+71.940232022" watchObservedRunningTime="2026-09-11 13:22:32.7243764 +0000 UTC m=+71.942913890"
Sep 11 13:22:59.432520 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:22:59.432468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:22:59.432934 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:22:59.432625    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:22:59.432934 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:22:59.432703    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls podName:934019ff-93c3-464b-906f-39c185f6d5e4 nodeName:}" failed. No retries permitted until 2026-09-11 13:24:03.432683241 +0000 UTC m=+162.651220720 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls") pod "dns-default-9wxh8" (UID: "934019ff-93c3-464b-906f-39c185f6d5e4") : secret "dns-default-metrics-tls" not found
Sep 11 13:23:01.785835 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.785798    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"]
Sep 11 13:23:01.789805 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.789789    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:01.796130 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.796089    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 11 13:23:01.796272 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.796146    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 11 13:23:01.796272 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.796154    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 13:23:01.796406 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.796392    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 13:23:01.796470 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.796435    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ldtdj\""
Sep 11 13:23:01.803009 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.801472    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"]
Sep 11 13:23:01.849627 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.849583    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-5wm6g\" (UID: \"b76062ff-6ab0-4d3a-b139-12b777cb0ce6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:01.849627 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.849628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-5wm6g\" (UID: \"b76062ff-6ab0-4d3a-b139-12b777cb0ce6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:01.849834 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.849652    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzn9b\" (UniqueName: \"kubernetes.io/projected/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-kube-api-access-pzn9b\") pod \"cluster-monitoring-operator-6cd954b897-5wm6g\" (UID: \"b76062ff-6ab0-4d3a-b139-12b777cb0ce6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:01.950981 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.950935    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-5wm6g\" (UID: \"b76062ff-6ab0-4d3a-b139-12b777cb0ce6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:01.950981 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.950983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-5wm6g\" (UID: \"b76062ff-6ab0-4d3a-b139-12b777cb0ce6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:01.951230 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:01.951096    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:23:01.951230 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:01.951171    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls podName:b76062ff-6ab0-4d3a-b139-12b777cb0ce6 nodeName:}" failed. No retries permitted until 2026-09-11 13:23:02.45115097 +0000 UTC m=+101.669688449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-5wm6g" (UID: "b76062ff-6ab0-4d3a-b139-12b777cb0ce6") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:23:01.951230 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.951189    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzn9b\" (UniqueName: \"kubernetes.io/projected/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-kube-api-access-pzn9b\") pod \"cluster-monitoring-operator-6cd954b897-5wm6g\" (UID: \"b76062ff-6ab0-4d3a-b139-12b777cb0ce6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:01.951758 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.951733    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-5wm6g\" (UID: \"b76062ff-6ab0-4d3a-b139-12b777cb0ce6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:01.960204 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:01.960179    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzn9b\" (UniqueName: \"kubernetes.io/projected/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-kube-api-access-pzn9b\") pod \"cluster-monitoring-operator-6cd954b897-5wm6g\" (UID: \"b76062ff-6ab0-4d3a-b139-12b777cb0ce6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:02.455595 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:02.455558    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-5wm6g\" (UID: \"b76062ff-6ab0-4d3a-b139-12b777cb0ce6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:02.455773 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:02.455678    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:23:02.455773 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:02.455745    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls podName:b76062ff-6ab0-4d3a-b139-12b777cb0ce6 nodeName:}" failed. No retries permitted until 2026-09-11 13:23:03.455728548 +0000 UTC m=+102.674266258 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-5wm6g" (UID: "b76062ff-6ab0-4d3a-b139-12b777cb0ce6") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:23:03.464172 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:03.464120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-5wm6g\" (UID: \"b76062ff-6ab0-4d3a-b139-12b777cb0ce6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:03.464666 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:03.464294    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:23:03.464666 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:03.464380    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls podName:b76062ff-6ab0-4d3a-b139-12b777cb0ce6 nodeName:}" failed. No retries permitted until 2026-09-11 13:23:05.464356353 +0000 UTC m=+104.682894129 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-5wm6g" (UID: "b76062ff-6ab0-4d3a-b139-12b777cb0ce6") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:23:03.661266 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:03.661232    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2d9t8"
Sep 11 13:23:05.477926 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.477861    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-5wm6g\" (UID: \"b76062ff-6ab0-4d3a-b139-12b777cb0ce6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:05.478428 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:05.478015    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:23:05.478428 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:05.478080    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls podName:b76062ff-6ab0-4d3a-b139-12b777cb0ce6 nodeName:}" failed. No retries permitted until 2026-09-11 13:23:09.478064291 +0000 UTC m=+108.696601761 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-5wm6g" (UID: "b76062ff-6ab0-4d3a-b139-12b777cb0ce6") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:23:05.681373 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.681336    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"]
Sep 11 13:23:05.713062 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.713031    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"
Sep 11 13:23:05.737762 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.737677    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:23:05.737762 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.737714    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rf47d\""
Sep 11 13:23:05.737989 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.737920    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 11 13:23:05.738061 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.738046    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 11 13:23:05.780951 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.780923    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6xn7l\" (UID: \"e0e189cb-554d-4207-86a4-e63f8c680b30\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"
Sep 11 13:23:05.780951 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.780952    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqg82\" (UniqueName: \"kubernetes.io/projected/e0e189cb-554d-4207-86a4-e63f8c680b30-kube-api-access-gqg82\") pod \"cluster-samples-operator-86499c89c4-6xn7l\" (UID: \"e0e189cb-554d-4207-86a4-e63f8c680b30\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"
Sep 11 13:23:05.797146 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.797118    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"]
Sep 11 13:23:05.846472 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.846436    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-9xvg6"]
Sep 11 13:23:05.862347 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.862316    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-9xvg6"
Sep 11 13:23:05.866144 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.866105    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-9xvg6"]
Sep 11 13:23:05.870118 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.870095    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 11 13:23:05.871291 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.871274    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:23:05.872188 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.872173    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-jjqwr\""
Sep 11 13:23:05.882062 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.882041    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6xn7l\" (UID: \"e0e189cb-554d-4207-86a4-e63f8c680b30\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"
Sep 11 13:23:05.882124 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.882069    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqg82\" (UniqueName: \"kubernetes.io/projected/e0e189cb-554d-4207-86a4-e63f8c680b30-kube-api-access-gqg82\") pod \"cluster-samples-operator-86499c89c4-6xn7l\" (UID: \"e0e189cb-554d-4207-86a4-e63f8c680b30\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"
Sep 11 13:23:05.882204 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:05.882188    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:23:05.882255 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:05.882246    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls podName:e0e189cb-554d-4207-86a4-e63f8c680b30 nodeName:}" failed. No retries permitted until 2026-09-11 13:23:06.382231066 +0000 UTC m=+105.600768540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-6xn7l" (UID: "e0e189cb-554d-4207-86a4-e63f8c680b30") : secret "samples-operator-tls" not found
Sep 11 13:23:05.896240 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.896214    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqg82\" (UniqueName: \"kubernetes.io/projected/e0e189cb-554d-4207-86a4-e63f8c680b30-kube-api-access-gqg82\") pod \"cluster-samples-operator-86499c89c4-6xn7l\" (UID: \"e0e189cb-554d-4207-86a4-e63f8c680b30\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"
Sep 11 13:23:05.982576 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.982542    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-85bpc"]
Sep 11 13:23:05.982744 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:05.982718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4twhb\" (UniqueName: \"kubernetes.io/projected/22456798-e0ae-4e71-94a9-08812435150a-kube-api-access-4twhb\") pod \"volume-data-source-validator-86695b7cc7-9xvg6\" (UID: \"22456798-e0ae-4e71-94a9-08812435150a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-9xvg6"
Sep 11 13:23:06.001836 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.001756    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-85bpc"]
Sep 11 13:23:06.001995 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.001875    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.004998 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.004963    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 13:23:06.005999 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.005962    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 13:23:06.006148 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.006018    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 13:23:06.006388 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.006368    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kd6rn\""
Sep 11 13:23:06.008218 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.008200    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 13:23:06.083367 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.083330    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4twhb\" (UniqueName: \"kubernetes.io/projected/22456798-e0ae-4e71-94a9-08812435150a-kube-api-access-4twhb\") pod \"volume-data-source-validator-86695b7cc7-9xvg6\" (UID: \"22456798-e0ae-4e71-94a9-08812435150a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-9xvg6"
Sep 11 13:23:06.083516 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.083395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/09ffbcb6-3382-48ea-9c3a-04cf17da047e-crio-socket\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.083516 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.083428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.083516 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.083444    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.083516 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.083460    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5dtd\" (UniqueName: \"kubernetes.io/projected/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-api-access-t5dtd\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.083516 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.083481    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/09ffbcb6-3382-48ea-9c3a-04cf17da047e-data-volume\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.092547 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.092522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4twhb\" (UniqueName: \"kubernetes.io/projected/22456798-e0ae-4e71-94a9-08812435150a-kube-api-access-4twhb\") pod \"volume-data-source-validator-86695b7cc7-9xvg6\" (UID: \"22456798-e0ae-4e71-94a9-08812435150a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-9xvg6"
Sep 11 13:23:06.171789 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.171749    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-9xvg6"
Sep 11 13:23:06.184126 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.184093    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/09ffbcb6-3382-48ea-9c3a-04cf17da047e-crio-socket\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.184251 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.184149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.184251 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.184177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.184251 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.184199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5dtd\" (UniqueName: \"kubernetes.io/projected/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-api-access-t5dtd\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.184251 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.184219    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/09ffbcb6-3382-48ea-9c3a-04cf17da047e-crio-socket\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.184251 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.184233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/09ffbcb6-3382-48ea-9c3a-04cf17da047e-data-volume\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.184499 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:06.184297    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:23:06.184499 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:06.184334    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:06.184499 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:06.184389    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm podName:09ffbcb6-3382-48ea-9c3a-04cf17da047e nodeName:}" failed. No retries permitted until 2026-09-11 13:23:06.684368374 +0000 UTC m=+105.902905849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-85bpc" (UID: "09ffbcb6-3382-48ea-9c3a-04cf17da047e") : configmap "kube-rbac-proxy" not found
Sep 11 13:23:06.184499 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:06.184409    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls podName:09ffbcb6-3382-48ea-9c3a-04cf17da047e nodeName:}" failed. No retries permitted until 2026-09-11 13:23:06.684399575 +0000 UTC m=+105.902937045 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-85bpc" (UID: "09ffbcb6-3382-48ea-9c3a-04cf17da047e") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:06.184670 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.184556    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/09ffbcb6-3382-48ea-9c3a-04cf17da047e-data-volume\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.193657 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.193637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5dtd\" (UniqueName: \"kubernetes.io/projected/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-api-access-t5dtd\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.386444 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.386413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6xn7l\" (UID: \"e0e189cb-554d-4207-86a4-e63f8c680b30\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"
Sep 11 13:23:06.386623 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:06.386545    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:23:06.386623 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:06.386611    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls podName:e0e189cb-554d-4207-86a4-e63f8c680b30 nodeName:}" failed. No retries permitted until 2026-09-11 13:23:07.386595818 +0000 UTC m=+106.605133300 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-6xn7l" (UID: "e0e189cb-554d-4207-86a4-e63f8c680b30") : secret "samples-operator-tls" not found
Sep 11 13:23:06.454674 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.454637    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-9xvg6"]
Sep 11 13:23:06.458083 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:23:06.458056    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22456798_e0ae_4e71_94a9_08812435150a.slice/crio-8eb12547bbea3c4f6e159fde3caae7e70d48bf8fcbd1eb26e3b84818e8764245 WatchSource:0}: Error finding container 8eb12547bbea3c4f6e159fde3caae7e70d48bf8fcbd1eb26e3b84818e8764245: Status 404 returned error can't find the container with id 8eb12547bbea3c4f6e159fde3caae7e70d48bf8fcbd1eb26e3b84818e8764245
Sep 11 13:23:06.679640 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.679560    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-mjvgv"]
Sep 11 13:23:06.688735 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.688709    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.688803 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.688745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:06.688849 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:06.688831    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:23:06.688884 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:06.688855    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:06.688932 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:06.688894    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm podName:09ffbcb6-3382-48ea-9c3a-04cf17da047e nodeName:}" failed. No retries permitted until 2026-09-11 13:23:07.688878709 +0000 UTC m=+106.907416178 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-85bpc" (UID: "09ffbcb6-3382-48ea-9c3a-04cf17da047e") : configmap "kube-rbac-proxy" not found
Sep 11 13:23:06.688932 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:06.688925    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls podName:09ffbcb6-3382-48ea-9c3a-04cf17da047e nodeName:}" failed. No retries permitted until 2026-09-11 13:23:07.688915047 +0000 UTC m=+106.907452518 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-85bpc" (UID: "09ffbcb6-3382-48ea-9c3a-04cf17da047e") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:06.715114 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.715084    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-mjvgv"]
Sep 11 13:23:06.715250 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.715176    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:06.718183 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.718155    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:23:06.718183 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.718179    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 11 13:23:06.718384 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.718194    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 11 13:23:06.718579 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.718562    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 11 13:23:06.718655 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.718612    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-fqdzp\""
Sep 11 13:23:06.726188 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.726170    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 11 13:23:06.741142 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.741110    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-9xvg6" event={"ID":"22456798-e0ae-4e71-94a9-08812435150a","Type":"ContainerStarted","Data":"8eb12547bbea3c4f6e159fde3caae7e70d48bf8fcbd1eb26e3b84818e8764245"}
Sep 11 13:23:06.789578 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.789543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d5a874c-26f3-4238-91aa-70bb9c2938ae-trusted-ca\") pod \"console-operator-696884fb9c-mjvgv\" (UID: \"9d5a874c-26f3-4238-91aa-70bb9c2938ae\") " pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:06.789754 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.789581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d5a874c-26f3-4238-91aa-70bb9c2938ae-serving-cert\") pod \"console-operator-696884fb9c-mjvgv\" (UID: \"9d5a874c-26f3-4238-91aa-70bb9c2938ae\") " pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:06.789754 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.789659    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d5a874c-26f3-4238-91aa-70bb9c2938ae-config\") pod \"console-operator-696884fb9c-mjvgv\" (UID: \"9d5a874c-26f3-4238-91aa-70bb9c2938ae\") " pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:06.789754 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.789691    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44bg4\" (UniqueName: \"kubernetes.io/projected/9d5a874c-26f3-4238-91aa-70bb9c2938ae-kube-api-access-44bg4\") pod \"console-operator-696884fb9c-mjvgv\" (UID: \"9d5a874c-26f3-4238-91aa-70bb9c2938ae\") " pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:06.890868 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.890831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d5a874c-26f3-4238-91aa-70bb9c2938ae-trusted-ca\") pod \"console-operator-696884fb9c-mjvgv\" (UID: \"9d5a874c-26f3-4238-91aa-70bb9c2938ae\") " pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:06.890868 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.890867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d5a874c-26f3-4238-91aa-70bb9c2938ae-serving-cert\") pod \"console-operator-696884fb9c-mjvgv\" (UID: \"9d5a874c-26f3-4238-91aa-70bb9c2938ae\") " pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:06.891159 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.890935    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d5a874c-26f3-4238-91aa-70bb9c2938ae-config\") pod \"console-operator-696884fb9c-mjvgv\" (UID: \"9d5a874c-26f3-4238-91aa-70bb9c2938ae\") " pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:06.891159 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.890970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44bg4\" (UniqueName: \"kubernetes.io/projected/9d5a874c-26f3-4238-91aa-70bb9c2938ae-kube-api-access-44bg4\") pod \"console-operator-696884fb9c-mjvgv\" (UID: \"9d5a874c-26f3-4238-91aa-70bb9c2938ae\") " pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:06.891714 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.891688    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d5a874c-26f3-4238-91aa-70bb9c2938ae-config\") pod \"console-operator-696884fb9c-mjvgv\" (UID: \"9d5a874c-26f3-4238-91aa-70bb9c2938ae\") " pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:06.891825 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.891794    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d5a874c-26f3-4238-91aa-70bb9c2938ae-trusted-ca\") pod \"console-operator-696884fb9c-mjvgv\" (UID: \"9d5a874c-26f3-4238-91aa-70bb9c2938ae\") " pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:06.893336 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.893320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d5a874c-26f3-4238-91aa-70bb9c2938ae-serving-cert\") pod \"console-operator-696884fb9c-mjvgv\" (UID: \"9d5a874c-26f3-4238-91aa-70bb9c2938ae\") " pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:06.901545 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:06.901522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44bg4\" (UniqueName: \"kubernetes.io/projected/9d5a874c-26f3-4238-91aa-70bb9c2938ae-kube-api-access-44bg4\") pod \"console-operator-696884fb9c-mjvgv\" (UID: \"9d5a874c-26f3-4238-91aa-70bb9c2938ae\") " pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:07.025024 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:07.024934    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:07.317160 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:07.317133    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-mjvgv"]
Sep 11 13:23:07.395932 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:07.395877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6xn7l\" (UID: \"e0e189cb-554d-4207-86a4-e63f8c680b30\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"
Sep 11 13:23:07.396103 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:07.396038    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:23:07.396167 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:07.396108    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls podName:e0e189cb-554d-4207-86a4-e63f8c680b30 nodeName:}" failed. No retries permitted until 2026-09-11 13:23:09.396093049 +0000 UTC m=+108.614630518 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-6xn7l" (UID: "e0e189cb-554d-4207-86a4-e63f8c680b30") : secret "samples-operator-tls" not found
Sep 11 13:23:07.525204 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:23:07.525173    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d5a874c_26f3_4238_91aa_70bb9c2938ae.slice/crio-4f122870ca51a3f39281c6b353baf04b8099cd541f3b6ff24954c9cd0864d9c7 WatchSource:0}: Error finding container 4f122870ca51a3f39281c6b353baf04b8099cd541f3b6ff24954c9cd0864d9c7: Status 404 returned error can't find the container with id 4f122870ca51a3f39281c6b353baf04b8099cd541f3b6ff24954c9cd0864d9c7
Sep 11 13:23:07.525479 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:07.525450    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rkrhj_030910bd-d1c8-48c5-bc44-5e561b3fbf60/dns-node-resolver/0.log"
Sep 11 13:23:07.698258 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:07.698209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:07.698258 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:07.698256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:07.698695 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:07.698339    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:23:07.698695 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:07.698369    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:07.698695 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:07.698414    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm podName:09ffbcb6-3382-48ea-9c3a-04cf17da047e nodeName:}" failed. No retries permitted until 2026-09-11 13:23:09.69838912 +0000 UTC m=+108.916926590 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-85bpc" (UID: "09ffbcb6-3382-48ea-9c3a-04cf17da047e") : configmap "kube-rbac-proxy" not found
Sep 11 13:23:07.698695 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:07.698428    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls podName:09ffbcb6-3382-48ea-9c3a-04cf17da047e nodeName:}" failed. No retries permitted until 2026-09-11 13:23:09.698422376 +0000 UTC m=+108.916959845 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-85bpc" (UID: "09ffbcb6-3382-48ea-9c3a-04cf17da047e") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:07.744605 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:07.744578    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-9xvg6" event={"ID":"22456798-e0ae-4e71-94a9-08812435150a","Type":"ContainerStarted","Data":"7d747414e87ec35372f092674e14bc4c5e1be4bab49cb97a8534d76f320f9070"}
Sep 11 13:23:07.745752 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:07.745729    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" event={"ID":"9d5a874c-26f3-4238-91aa-70bb9c2938ae","Type":"ContainerStarted","Data":"4f122870ca51a3f39281c6b353baf04b8099cd541f3b6ff24954c9cd0864d9c7"}
Sep 11 13:23:07.780334 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:07.780287    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-9xvg6" podStartSLOduration=1.689060649 podStartE2EDuration="2.780273166s" totalImagesPullingTime="1.091212517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:23:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:07.779817613 +0000 UTC m=+106.998355107" watchObservedRunningTime="2026-09-11 13:23:07.780273166 +0000 UTC m=+106.998810657"
Sep 11 13:23:08.524823 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:08.524791    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gz6tv_c3b3d505-6502-4cc5-9aec-aef4cd996049/node-ca/0.log"
Sep 11 13:23:09.129353 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:09.129315    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5zw59_fc1353d3-7bc6-45bd-9ecf-c325b4469fb8/serve-healthcheck-canary/0.log"
Sep 11 13:23:09.412194 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:09.412099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6xn7l\" (UID: \"e0e189cb-554d-4207-86a4-e63f8c680b30\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"
Sep 11 13:23:09.412338 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:09.412283    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:23:09.412400 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:09.412361    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls podName:e0e189cb-554d-4207-86a4-e63f8c680b30 nodeName:}" failed. No retries permitted until 2026-09-11 13:23:13.412344457 +0000 UTC m=+112.630881927 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-6xn7l" (UID: "e0e189cb-554d-4207-86a4-e63f8c680b30") : secret "samples-operator-tls" not found
Sep 11 13:23:09.512959 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:09.512915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-5wm6g\" (UID: \"b76062ff-6ab0-4d3a-b139-12b777cb0ce6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:09.513159 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:09.513069    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:23:09.513226 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:09.513185    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls podName:b76062ff-6ab0-4d3a-b139-12b777cb0ce6 nodeName:}" failed. No retries permitted until 2026-09-11 13:23:17.513161632 +0000 UTC m=+116.731699117 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-5wm6g" (UID: "b76062ff-6ab0-4d3a-b139-12b777cb0ce6") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:23:09.714347 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:09.714319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:09.714439 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:09.714363    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:09.714439 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:09.714428    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:23:09.714503 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:09.714495    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm podName:09ffbcb6-3382-48ea-9c3a-04cf17da047e nodeName:}" failed. No retries permitted until 2026-09-11 13:23:13.714475856 +0000 UTC m=+112.933013339 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-85bpc" (UID: "09ffbcb6-3382-48ea-9c3a-04cf17da047e") : configmap "kube-rbac-proxy" not found
Sep 11 13:23:09.714543 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:09.714520    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:09.714575 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:09.714552    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls podName:09ffbcb6-3382-48ea-9c3a-04cf17da047e nodeName:}" failed. No retries permitted until 2026-09-11 13:23:13.714543844 +0000 UTC m=+112.933081317 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-85bpc" (UID: "09ffbcb6-3382-48ea-9c3a-04cf17da047e") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:09.753210 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:09.753179    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" event={"ID":"9d5a874c-26f3-4238-91aa-70bb9c2938ae","Type":"ContainerStarted","Data":"600b284b32fad2f334161ca8398e6093429b842adb8289aae550ba8c7ed73bfd"}
Sep 11 13:23:09.753371 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:09.753339    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:09.754751 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:09.754729    2580 patch_prober.go:28] interesting pod/console-operator-696884fb9c-mjvgv container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.133.0.12:8443/readyz\": dial tcp 10.133.0.12:8443: connect: connection refused" start-of-body=
Sep 11 13:23:09.754800 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:09.754774    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" podUID="9d5a874c-26f3-4238-91aa-70bb9c2938ae" containerName="console-operator" probeResult="failure" output="Get \"https://10.133.0.12:8443/readyz\": dial tcp 10.133.0.12:8443: connect: connection refused"
Sep 11 13:23:09.772023 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:09.771981    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" podStartSLOduration=1.652425799 podStartE2EDuration="3.771966808s" totalImagesPullingTime="2.119541009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:23:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:09.771410289 +0000 UTC m=+108.989947781" watchObservedRunningTime="2026-09-11 13:23:09.771966808 +0000 UTC m=+108.990504300"
Sep 11 13:23:10.628198 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.628167    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f"]
Sep 11 13:23:10.630047 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.630031    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f"
Sep 11 13:23:10.632842 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.632819    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 11 13:23:10.635802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.635771    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 11 13:23:10.636861 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.636824    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:23:10.636984 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.636869    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-l2sk9\""
Sep 11 13:23:10.636984 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.636887    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 11 13:23:10.641617 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.641596    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f"]
Sep 11 13:23:10.723624 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.723591    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv9qr\" (UniqueName: \"kubernetes.io/projected/fa2dddcb-d99b-465d-be8f-d9db18b9555c-kube-api-access-xv9qr\") pod \"kube-storage-version-migrator-operator-6c8f84f897-kjs8f\" (UID: \"fa2dddcb-d99b-465d-be8f-d9db18b9555c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f"
Sep 11 13:23:10.723820 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.723656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fa2dddcb-d99b-465d-be8f-d9db18b9555c-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-kjs8f\" (UID: \"fa2dddcb-d99b-465d-be8f-d9db18b9555c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f"
Sep 11 13:23:10.723820 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.723743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fa2dddcb-d99b-465d-be8f-d9db18b9555c-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-kjs8f\" (UID: \"fa2dddcb-d99b-465d-be8f-d9db18b9555c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f"
Sep 11 13:23:10.757031 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.757007    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/0.log"
Sep 11 13:23:10.757164 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.757043    2580 generic.go:356] "Generic (PLEG): container finished" podID="9d5a874c-26f3-4238-91aa-70bb9c2938ae" containerID="600b284b32fad2f334161ca8398e6093429b842adb8289aae550ba8c7ed73bfd" exitCode=255
Sep 11 13:23:10.757164 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.757077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" event={"ID":"9d5a874c-26f3-4238-91aa-70bb9c2938ae","Type":"ContainerDied","Data":"600b284b32fad2f334161ca8398e6093429b842adb8289aae550ba8c7ed73bfd"}
Sep 11 13:23:10.757375 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.757362    2580 scope.go:117] "RemoveContainer" containerID="600b284b32fad2f334161ca8398e6093429b842adb8289aae550ba8c7ed73bfd"
Sep 11 13:23:10.824936 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.824895    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fa2dddcb-d99b-465d-be8f-d9db18b9555c-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-kjs8f\" (UID: \"fa2dddcb-d99b-465d-be8f-d9db18b9555c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f"
Sep 11 13:23:10.825125 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.825105    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv9qr\" (UniqueName: \"kubernetes.io/projected/fa2dddcb-d99b-465d-be8f-d9db18b9555c-kube-api-access-xv9qr\") pod \"kube-storage-version-migrator-operator-6c8f84f897-kjs8f\" (UID: \"fa2dddcb-d99b-465d-be8f-d9db18b9555c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f"
Sep 11 13:23:10.825188 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.825162    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fa2dddcb-d99b-465d-be8f-d9db18b9555c-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-kjs8f\" (UID: \"fa2dddcb-d99b-465d-be8f-d9db18b9555c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f"
Sep 11 13:23:10.825653 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.825633    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fa2dddcb-d99b-465d-be8f-d9db18b9555c-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-kjs8f\" (UID: \"fa2dddcb-d99b-465d-be8f-d9db18b9555c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f"
Sep 11 13:23:10.826864 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.826840    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fa2dddcb-d99b-465d-be8f-d9db18b9555c-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-kjs8f\" (UID: \"fa2dddcb-d99b-465d-be8f-d9db18b9555c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f"
Sep 11 13:23:10.834085 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.834062    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv9qr\" (UniqueName: \"kubernetes.io/projected/fa2dddcb-d99b-465d-be8f-d9db18b9555c-kube-api-access-xv9qr\") pod \"kube-storage-version-migrator-operator-6c8f84f897-kjs8f\" (UID: \"fa2dddcb-d99b-465d-be8f-d9db18b9555c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f"
Sep 11 13:23:10.939056 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:10.938972    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f"
Sep 11 13:23:11.210521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.210443    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f"]
Sep 11 13:23:11.213282 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:23:11.213250    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa2dddcb_d99b_465d_be8f_d9db18b9555c.slice/crio-d6577b6a82087963b4a6b5e415cf8b9e3c2880be52c304b7329fa50c3658e387 WatchSource:0}: Error finding container d6577b6a82087963b4a6b5e415cf8b9e3c2880be52c304b7329fa50c3658e387: Status 404 returned error can't find the container with id d6577b6a82087963b4a6b5e415cf8b9e3c2880be52c304b7329fa50c3658e387
Sep 11 13:23:11.651958 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.651923    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb"]
Sep 11 13:23:11.654862 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.654845    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb"
Sep 11 13:23:11.657541 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.657515    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-m949z\""
Sep 11 13:23:11.657634 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.657617    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 11 13:23:11.657756 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.657741    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:23:11.658608 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.658592    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 11 13:23:11.658677 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.658622    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 11 13:23:11.667289 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.667254    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb"]
Sep 11 13:23:11.734096 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.734057    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/24bb030f-2e95-4074-81ba-c28c6859db4c-config\") pod \"service-ca-operator-cf7c776b7-rvdfb\" (UID: \"24bb030f-2e95-4074-81ba-c28c6859db4c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb"
Sep 11 13:23:11.734264 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.734155    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24bb030f-2e95-4074-81ba-c28c6859db4c-serving-cert\") pod \"service-ca-operator-cf7c776b7-rvdfb\" (UID: \"24bb030f-2e95-4074-81ba-c28c6859db4c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb"
Sep 11 13:23:11.734264 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.734185    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9tl8\" (UniqueName: \"kubernetes.io/projected/24bb030f-2e95-4074-81ba-c28c6859db4c-kube-api-access-v9tl8\") pod \"service-ca-operator-cf7c776b7-rvdfb\" (UID: \"24bb030f-2e95-4074-81ba-c28c6859db4c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb"
Sep 11 13:23:11.762217 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.762191    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/1.log"
Sep 11 13:23:11.762592 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.762577    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/0.log"
Sep 11 13:23:11.762643 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.762611    2580 generic.go:356] "Generic (PLEG): container finished" podID="9d5a874c-26f3-4238-91aa-70bb9c2938ae" containerID="e55a66546811367b30656e3dcb3d007c8c7d70a66cba9ad62ad42b69d6179c53" exitCode=255
Sep 11 13:23:11.762699 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.762685    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" event={"ID":"9d5a874c-26f3-4238-91aa-70bb9c2938ae","Type":"ContainerDied","Data":"e55a66546811367b30656e3dcb3d007c8c7d70a66cba9ad62ad42b69d6179c53"}
Sep 11 13:23:11.762763 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.762750    2580 scope.go:117] "RemoveContainer" containerID="600b284b32fad2f334161ca8398e6093429b842adb8289aae550ba8c7ed73bfd"
Sep 11 13:23:11.762949 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.762934    2580 scope.go:117] "RemoveContainer" containerID="e55a66546811367b30656e3dcb3d007c8c7d70a66cba9ad62ad42b69d6179c53"
Sep 11 13:23:11.763139 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:11.763116    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-mjvgv_openshift-console-operator(9d5a874c-26f3-4238-91aa-70bb9c2938ae)\"" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" podUID="9d5a874c-26f3-4238-91aa-70bb9c2938ae"
Sep 11 13:23:11.764215 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.763983    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f" event={"ID":"fa2dddcb-d99b-465d-be8f-d9db18b9555c","Type":"ContainerStarted","Data":"d6577b6a82087963b4a6b5e415cf8b9e3c2880be52c304b7329fa50c3658e387"}
Sep 11 13:23:11.835210 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.835171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/24bb030f-2e95-4074-81ba-c28c6859db4c-config\") pod \"service-ca-operator-cf7c776b7-rvdfb\" (UID: \"24bb030f-2e95-4074-81ba-c28c6859db4c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb"
Sep 11 13:23:11.835375 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.835344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24bb030f-2e95-4074-81ba-c28c6859db4c-serving-cert\") pod \"service-ca-operator-cf7c776b7-rvdfb\" (UID: \"24bb030f-2e95-4074-81ba-c28c6859db4c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb"
Sep 11 13:23:11.835473 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.835455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9tl8\" (UniqueName: \"kubernetes.io/projected/24bb030f-2e95-4074-81ba-c28c6859db4c-kube-api-access-v9tl8\") pod \"service-ca-operator-cf7c776b7-rvdfb\" (UID: \"24bb030f-2e95-4074-81ba-c28c6859db4c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb"
Sep 11 13:23:11.835717 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.835696    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/24bb030f-2e95-4074-81ba-c28c6859db4c-config\") pod \"service-ca-operator-cf7c776b7-rvdfb\" (UID: \"24bb030f-2e95-4074-81ba-c28c6859db4c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb"
Sep 11 13:23:11.837518 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.837499    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24bb030f-2e95-4074-81ba-c28c6859db4c-serving-cert\") pod \"service-ca-operator-cf7c776b7-rvdfb\" (UID: \"24bb030f-2e95-4074-81ba-c28c6859db4c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb"
Sep 11 13:23:11.845604 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.845575    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9tl8\" (UniqueName: \"kubernetes.io/projected/24bb030f-2e95-4074-81ba-c28c6859db4c-kube-api-access-v9tl8\") pod \"service-ca-operator-cf7c776b7-rvdfb\" (UID: \"24bb030f-2e95-4074-81ba-c28c6859db4c\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb"
Sep 11 13:23:11.963318 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:11.963229    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb"
Sep 11 13:23:12.244234 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:12.244157    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb"]
Sep 11 13:23:12.247279 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:23:12.247250    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24bb030f_2e95_4074_81ba_c28c6859db4c.slice/crio-baa50c39e863835d2ea837f8d6abde4fe5795c402594518919514437398884c7 WatchSource:0}: Error finding container baa50c39e863835d2ea837f8d6abde4fe5795c402594518919514437398884c7: Status 404 returned error can't find the container with id baa50c39e863835d2ea837f8d6abde4fe5795c402594518919514437398884c7
Sep 11 13:23:12.768576 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:12.768548    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/1.log"
Sep 11 13:23:12.769031 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:12.769000    2580 scope.go:117] "RemoveContainer" containerID="e55a66546811367b30656e3dcb3d007c8c7d70a66cba9ad62ad42b69d6179c53"
Sep 11 13:23:12.769294 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:12.769182    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-mjvgv_openshift-console-operator(9d5a874c-26f3-4238-91aa-70bb9c2938ae)\"" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" podUID="9d5a874c-26f3-4238-91aa-70bb9c2938ae"
Sep 11 13:23:12.770034 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:12.770010    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb" event={"ID":"24bb030f-2e95-4074-81ba-c28c6859db4c","Type":"ContainerStarted","Data":"baa50c39e863835d2ea837f8d6abde4fe5795c402594518919514437398884c7"}
Sep 11 13:23:13.277082 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:13.277005    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-mtj8n"]
Sep 11 13:23:13.280518 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:13.280466    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-mtj8n"
Sep 11 13:23:13.283300 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:13.283274    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-f5bf4\""
Sep 11 13:23:13.287670 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:13.287546    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-mtj8n"]
Sep 11 13:23:13.448814 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:13.448776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6xn7l\" (UID: \"e0e189cb-554d-4207-86a4-e63f8c680b30\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"
Sep 11 13:23:13.449006 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:13.448871    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgxnm\" (UniqueName: \"kubernetes.io/projected/c4185171-6faa-4d22-9c47-2cc082cd113f-kube-api-access-jgxnm\") pod \"network-check-source-6f8b768455-mtj8n\" (UID: \"c4185171-6faa-4d22-9c47-2cc082cd113f\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-mtj8n"
Sep 11 13:23:13.449006 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:13.448977    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 13:23:13.449100 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:13.449076    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls podName:e0e189cb-554d-4207-86a4-e63f8c680b30 nodeName:}" failed. No retries permitted until 2026-09-11 13:23:21.449061316 +0000 UTC m=+120.667598804 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-6xn7l" (UID: "e0e189cb-554d-4207-86a4-e63f8c680b30") : secret "samples-operator-tls" not found
Sep 11 13:23:13.549587 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:13.549562    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgxnm\" (UniqueName: \"kubernetes.io/projected/c4185171-6faa-4d22-9c47-2cc082cd113f-kube-api-access-jgxnm\") pod \"network-check-source-6f8b768455-mtj8n\" (UID: \"c4185171-6faa-4d22-9c47-2cc082cd113f\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-mtj8n"
Sep 11 13:23:13.561292 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:13.561261    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgxnm\" (UniqueName: \"kubernetes.io/projected/c4185171-6faa-4d22-9c47-2cc082cd113f-kube-api-access-jgxnm\") pod \"network-check-source-6f8b768455-mtj8n\" (UID: \"c4185171-6faa-4d22-9c47-2cc082cd113f\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-mtj8n"
Sep 11 13:23:13.592104 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:13.592069    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-mtj8n"
Sep 11 13:23:13.751961 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:13.751921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:13.752169 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:13.752039    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:23:13.752169 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:13.752063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:13.752169 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:13.752117    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm podName:09ffbcb6-3382-48ea-9c3a-04cf17da047e nodeName:}" failed. No retries permitted until 2026-09-11 13:23:21.752097171 +0000 UTC m=+120.970634656 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-85bpc" (UID: "09ffbcb6-3382-48ea-9c3a-04cf17da047e") : configmap "kube-rbac-proxy" not found
Sep 11 13:23:13.752362 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:13.752182    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:13.752362 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:13.752239    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls podName:09ffbcb6-3382-48ea-9c3a-04cf17da047e nodeName:}" failed. No retries permitted until 2026-09-11 13:23:21.752223139 +0000 UTC m=+120.970760630 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-85bpc" (UID: "09ffbcb6-3382-48ea-9c3a-04cf17da047e") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:13.779747 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:13.779659    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f" event={"ID":"fa2dddcb-d99b-465d-be8f-d9db18b9555c","Type":"ContainerStarted","Data":"606a269b1bb2a597ef2eb6f9b00d492cb9d37440db927ca6fe48a2375076639d"}
Sep 11 13:23:13.798867 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:13.798812    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f" podStartSLOduration=1.5523148390000001 podStartE2EDuration="3.79878919s" totalImagesPullingTime="2.246474351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:23:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:13.797353294 +0000 UTC m=+113.015890790" watchObservedRunningTime="2026-09-11 13:23:13.79878919 +0000 UTC m=+113.017326687"
Sep 11 13:23:13.872773 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:13.872748    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-mtj8n"]
Sep 11 13:23:14.348446 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:23:14.348402    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4185171_6faa_4d22_9c47_2cc082cd113f.slice/crio-f2eed4cbdbf4213e054c8e9b45aa5340ae885e4b66b569edb486f3ab48edc5d7 WatchSource:0}: Error finding container f2eed4cbdbf4213e054c8e9b45aa5340ae885e4b66b569edb486f3ab48edc5d7: Status 404 returned error can't find the container with id f2eed4cbdbf4213e054c8e9b45aa5340ae885e4b66b569edb486f3ab48edc5d7
Sep 11 13:23:14.608291 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:14.608209    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-f54b4"]
Sep 11 13:23:14.611392 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:14.611371    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f54b4"
Sep 11 13:23:14.617329 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:14.616333    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 11 13:23:14.617329 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:14.616385    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-jwj8b\""
Sep 11 13:23:14.617329 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:14.616338    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 11 13:23:14.625838 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:14.625815    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-f54b4"]
Sep 11 13:23:14.762421 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:14.762394    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz265\" (UniqueName: \"kubernetes.io/projected/f5ae2e94-9288-43a4-a62a-e00a66539d9d-kube-api-access-tz265\") pod \"migrator-8b4d697c5-f54b4\" (UID: \"f5ae2e94-9288-43a4-a62a-e00a66539d9d\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f54b4"
Sep 11 13:23:14.784572 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:14.784538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb" event={"ID":"24bb030f-2e95-4074-81ba-c28c6859db4c","Type":"ContainerStarted","Data":"8adef6b23ca51b95f021147a63aa63ce0a9c3de78ad2cf059487ee90872bb0c2"}
Sep 11 13:23:14.786212 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:14.786183    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-mtj8n" event={"ID":"c4185171-6faa-4d22-9c47-2cc082cd113f","Type":"ContainerStarted","Data":"ecafd2d22a5f299c83dfa57309e9ecda0cb9abf3ab96cf12af87a7260d03e65c"}
Sep 11 13:23:14.786325 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:14.786223    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-mtj8n" event={"ID":"c4185171-6faa-4d22-9c47-2cc082cd113f","Type":"ContainerStarted","Data":"f2eed4cbdbf4213e054c8e9b45aa5340ae885e4b66b569edb486f3ab48edc5d7"}
Sep 11 13:23:14.802076 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:14.802020    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb" podStartSLOduration=1.651503827 podStartE2EDuration="3.802001748s" totalImagesPullingTime="2.150497921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:23:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:14.801309557 +0000 UTC m=+114.019847050" watchObservedRunningTime="2026-09-11 13:23:14.802001748 +0000 UTC m=+114.020539241"
Sep 11 13:23:14.817331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:14.817279    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-mtj8n" podStartSLOduration=1.8172611170000001 podStartE2EDuration="1.817261117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:23:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:14.816864112 +0000 UTC m=+114.035401602" watchObservedRunningTime="2026-09-11 13:23:14.817261117 +0000 UTC m=+114.035798606"
Sep 11 13:23:14.863202 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:14.863119    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz265\" (UniqueName: \"kubernetes.io/projected/f5ae2e94-9288-43a4-a62a-e00a66539d9d-kube-api-access-tz265\") pod \"migrator-8b4d697c5-f54b4\" (UID: \"f5ae2e94-9288-43a4-a62a-e00a66539d9d\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f54b4"
Sep 11 13:23:14.872011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:14.871983    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz265\" (UniqueName: \"kubernetes.io/projected/f5ae2e94-9288-43a4-a62a-e00a66539d9d-kube-api-access-tz265\") pod \"migrator-8b4d697c5-f54b4\" (UID: \"f5ae2e94-9288-43a4-a62a-e00a66539d9d\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f54b4"
Sep 11 13:23:14.920864 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:14.920830    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f54b4"
Sep 11 13:23:15.201444 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:15.201360    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-f54b4"]
Sep 11 13:23:15.205080 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:23:15.205037    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ae2e94_9288_43a4_a62a_e00a66539d9d.slice/crio-472d1c354717f3b4ed83f64a3e273165e3112c2b82cbf0d26bd2c9a0c7cb846b WatchSource:0}: Error finding container 472d1c354717f3b4ed83f64a3e273165e3112c2b82cbf0d26bd2c9a0c7cb846b: Status 404 returned error can't find the container with id 472d1c354717f3b4ed83f64a3e273165e3112c2b82cbf0d26bd2c9a0c7cb846b
Sep 11 13:23:15.792385 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:15.792342    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f54b4" event={"ID":"f5ae2e94-9288-43a4-a62a-e00a66539d9d","Type":"ContainerStarted","Data":"472d1c354717f3b4ed83f64a3e273165e3112c2b82cbf0d26bd2c9a0c7cb846b"}
Sep 11 13:23:16.796973 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:16.796863    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f54b4" event={"ID":"f5ae2e94-9288-43a4-a62a-e00a66539d9d","Type":"ContainerStarted","Data":"ac03ccbe45478788d65aeeecc572d5e78be21837ef5f4ee9ec6ffefd122e8c7c"}
Sep 11 13:23:16.796973 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:16.796939    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f54b4" event={"ID":"f5ae2e94-9288-43a4-a62a-e00a66539d9d","Type":"ContainerStarted","Data":"127597dc4452866d6119d20e4582634d2a460a3fe3d38ec28330666b189e455f"}
Sep 11 13:23:16.818919 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:16.818867    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f54b4" podStartSLOduration=1.683914912 podStartE2EDuration="2.818852643s" totalImagesPullingTime="1.134937731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:23:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:16.817267414 +0000 UTC m=+116.035804905" watchObservedRunningTime="2026-09-11 13:23:16.818852643 +0000 UTC m=+116.037390170"
Sep 11 13:23:17.025769 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:17.025733    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:17.026136 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:17.026123    2580 scope.go:117] "RemoveContainer" containerID="e55a66546811367b30656e3dcb3d007c8c7d70a66cba9ad62ad42b69d6179c53"
Sep 11 13:23:17.026290 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:17.026272    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-mjvgv_openshift-console-operator(9d5a874c-26f3-4238-91aa-70bb9c2938ae)\"" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" podUID="9d5a874c-26f3-4238-91aa-70bb9c2938ae"
Sep 11 13:23:17.588015 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:17.587974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-5wm6g\" (UID: \"b76062ff-6ab0-4d3a-b139-12b777cb0ce6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:17.588203 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:17.588158    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:23:17.588260 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:17.588239    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls podName:b76062ff-6ab0-4d3a-b139-12b777cb0ce6 nodeName:}" failed. No retries permitted until 2026-09-11 13:23:33.58821676 +0000 UTC m=+132.806754256 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-5wm6g" (UID: "b76062ff-6ab0-4d3a-b139-12b777cb0ce6") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:23:18.406920 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.406883    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-c9922"]
Sep 11 13:23:18.409820 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.409800    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-c9922"
Sep 11 13:23:18.412416 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.412396    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 11 13:23:18.413670 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.413639    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 11 13:23:18.413670 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.413651    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 11 13:23:18.413817 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.413677    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 11 13:23:18.413817 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.413651    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-74jb6\""
Sep 11 13:23:18.418813 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.418791    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-c9922"]
Sep 11 13:23:18.596222 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.596183    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj7wv\" (UniqueName: \"kubernetes.io/projected/f3b759a1-09c7-4707-ab65-877b765b7837-kube-api-access-xj7wv\") pod \"service-ca-568869fdc5-c9922\" (UID: \"f3b759a1-09c7-4707-ab65-877b765b7837\") " pod="openshift-service-ca/service-ca-568869fdc5-c9922"
Sep 11 13:23:18.596383 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.596323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f3b759a1-09c7-4707-ab65-877b765b7837-signing-cabundle\") pod \"service-ca-568869fdc5-c9922\" (UID: \"f3b759a1-09c7-4707-ab65-877b765b7837\") " pod="openshift-service-ca/service-ca-568869fdc5-c9922"
Sep 11 13:23:18.596383 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.596363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f3b759a1-09c7-4707-ab65-877b765b7837-signing-key\") pod \"service-ca-568869fdc5-c9922\" (UID: \"f3b759a1-09c7-4707-ab65-877b765b7837\") " pod="openshift-service-ca/service-ca-568869fdc5-c9922"
Sep 11 13:23:18.697372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.697288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f3b759a1-09c7-4707-ab65-877b765b7837-signing-cabundle\") pod \"service-ca-568869fdc5-c9922\" (UID: \"f3b759a1-09c7-4707-ab65-877b765b7837\") " pod="openshift-service-ca/service-ca-568869fdc5-c9922"
Sep 11 13:23:18.697372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.697330    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f3b759a1-09c7-4707-ab65-877b765b7837-signing-key\") pod \"service-ca-568869fdc5-c9922\" (UID: \"f3b759a1-09c7-4707-ab65-877b765b7837\") " pod="openshift-service-ca/service-ca-568869fdc5-c9922"
Sep 11 13:23:18.697372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.697360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj7wv\" (UniqueName: \"kubernetes.io/projected/f3b759a1-09c7-4707-ab65-877b765b7837-kube-api-access-xj7wv\") pod \"service-ca-568869fdc5-c9922\" (UID: \"f3b759a1-09c7-4707-ab65-877b765b7837\") " pod="openshift-service-ca/service-ca-568869fdc5-c9922"
Sep 11 13:23:18.698071 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.698046    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f3b759a1-09c7-4707-ab65-877b765b7837-signing-cabundle\") pod \"service-ca-568869fdc5-c9922\" (UID: \"f3b759a1-09c7-4707-ab65-877b765b7837\") " pod="openshift-service-ca/service-ca-568869fdc5-c9922"
Sep 11 13:23:18.699647 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.699626    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f3b759a1-09c7-4707-ab65-877b765b7837-signing-key\") pod \"service-ca-568869fdc5-c9922\" (UID: \"f3b759a1-09c7-4707-ab65-877b765b7837\") " pod="openshift-service-ca/service-ca-568869fdc5-c9922"
Sep 11 13:23:18.706771 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.706749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj7wv\" (UniqueName: \"kubernetes.io/projected/f3b759a1-09c7-4707-ab65-877b765b7837-kube-api-access-xj7wv\") pod \"service-ca-568869fdc5-c9922\" (UID: \"f3b759a1-09c7-4707-ab65-877b765b7837\") " pod="openshift-service-ca/service-ca-568869fdc5-c9922"
Sep 11 13:23:18.718752 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.718730    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-c9922"
Sep 11 13:23:18.990570 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:18.990493    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-c9922"]
Sep 11 13:23:18.993616 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:23:18.993591    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3b759a1_09c7_4707_ab65_877b765b7837.slice/crio-2ebc9479f4ca1ac70696d7e8060452d3bbafc8eb4a2fca2b45c825a756c1f5f5 WatchSource:0}: Error finding container 2ebc9479f4ca1ac70696d7e8060452d3bbafc8eb4a2fca2b45c825a756c1f5f5: Status 404 returned error can't find the container with id 2ebc9479f4ca1ac70696d7e8060452d3bbafc8eb4a2fca2b45c825a756c1f5f5
Sep 11 13:23:19.753840 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:19.753805    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:19.754326 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:19.754309    2580 scope.go:117] "RemoveContainer" containerID="e55a66546811367b30656e3dcb3d007c8c7d70a66cba9ad62ad42b69d6179c53"
Sep 11 13:23:19.754507 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:19.754485    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-mjvgv_openshift-console-operator(9d5a874c-26f3-4238-91aa-70bb9c2938ae)\"" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" podUID="9d5a874c-26f3-4238-91aa-70bb9c2938ae"
Sep 11 13:23:19.808028 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:19.807992    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-c9922" event={"ID":"f3b759a1-09c7-4707-ab65-877b765b7837","Type":"ContainerStarted","Data":"eff41bcba54ed2fbc531411ecffb07b94d5d7654a2779252a0a6c60de0c7961d"}
Sep 11 13:23:19.808210 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:19.808047    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-c9922" event={"ID":"f3b759a1-09c7-4707-ab65-877b765b7837","Type":"ContainerStarted","Data":"2ebc9479f4ca1ac70696d7e8060452d3bbafc8eb4a2fca2b45c825a756c1f5f5"}
Sep 11 13:23:19.827345 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:19.827293    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-c9922" podStartSLOduration=1.827272966 podStartE2EDuration="1.827272966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:23:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:19.826803233 +0000 UTC m=+119.045340726" watchObservedRunningTime="2026-09-11 13:23:19.827272966 +0000 UTC m=+119.045810458"
Sep 11 13:23:21.519644 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:21.519599    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6xn7l\" (UID: \"e0e189cb-554d-4207-86a4-e63f8c680b30\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"
Sep 11 13:23:21.521955 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:21.521931    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0e189cb-554d-4207-86a4-e63f8c680b30-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6xn7l\" (UID: \"e0e189cb-554d-4207-86a4-e63f8c680b30\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"
Sep 11 13:23:21.625071 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:21.625037    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rf47d\""
Sep 11 13:23:21.633142 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:21.633117    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"
Sep 11 13:23:21.822312 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:21.822282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:21.822312 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:21.822315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:21.822492 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:21.822433    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:21.822492 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:21.822482    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls podName:09ffbcb6-3382-48ea-9c3a-04cf17da047e nodeName:}" failed. No retries permitted until 2026-09-11 13:23:37.822468636 +0000 UTC m=+137.041006106 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-85bpc" (UID: "09ffbcb6-3382-48ea-9c3a-04cf17da047e") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:21.822837 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:21.822814    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09ffbcb6-3382-48ea-9c3a-04cf17da047e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:21.913259 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:21.913097    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l"]
Sep 11 13:23:22.819441 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:22.819363    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l" event={"ID":"e0e189cb-554d-4207-86a4-e63f8c680b30","Type":"ContainerStarted","Data":"5b9289d95c0d517058390cfef30258c4ecb220a26418953d9ad863fbb572ad63"}
Sep 11 13:23:24.828454 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:24.828413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l" event={"ID":"e0e189cb-554d-4207-86a4-e63f8c680b30","Type":"ContainerStarted","Data":"ef667b28779a9430be01354fd00dc1fcbf9748fb4d65428fb978e9281197bfa7"}
Sep 11 13:23:24.828778 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:24.828463    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l" event={"ID":"e0e189cb-554d-4207-86a4-e63f8c680b30","Type":"ContainerStarted","Data":"9abd40fbc1d572ed1b1a081681a0c5c5ee942220173221e6f868996b22badebc"}
Sep 11 13:23:24.864346 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:24.864288    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6xn7l" podStartSLOduration=17.739108855 podStartE2EDuration="19.864274774s" totalImagesPullingTime="2.125165919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:23:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:24.862897182 +0000 UTC m=+124.081434673" watchObservedRunningTime="2026-09-11 13:23:24.864274774 +0000 UTC m=+124.082812266"
Sep 11 13:23:31.100413 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:31.100368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs\") pod \"network-metrics-daemon-hxs7m\" (UID: \"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1\") " pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:23:31.102672 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:31.102643    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dcd97e-d15b-4ca5-ac15-997c0ba69cf1-metrics-certs\") pod \"network-metrics-daemon-hxs7m\" (UID: \"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1\") " pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:23:31.237372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:31.237339    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j2l67\""
Sep 11 13:23:31.245525 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:31.245508    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hxs7m"
Sep 11 13:23:31.513270 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:31.513090    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hxs7m"]
Sep 11 13:23:31.515636 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:23:31.515610    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30dcd97e_d15b_4ca5_ac15_997c0ba69cf1.slice/crio-c3247a6f0b3adf54ba689d27ea89f69c311d9e060485820271363490e4eb352c WatchSource:0}: Error finding container c3247a6f0b3adf54ba689d27ea89f69c311d9e060485820271363490e4eb352c: Status 404 returned error can't find the container with id c3247a6f0b3adf54ba689d27ea89f69c311d9e060485820271363490e4eb352c
Sep 11 13:23:31.857413 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:31.857376    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hxs7m" event={"ID":"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1","Type":"ContainerStarted","Data":"c3247a6f0b3adf54ba689d27ea89f69c311d9e060485820271363490e4eb352c"}
Sep 11 13:23:32.310345 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:32.310255    2580 scope.go:117] "RemoveContainer" containerID="e55a66546811367b30656e3dcb3d007c8c7d70a66cba9ad62ad42b69d6179c53"
Sep 11 13:23:32.862976 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:32.862945    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hxs7m" event={"ID":"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1","Type":"ContainerStarted","Data":"ee50219ee503c5d796c76ba1cd3551159c55ca337b2144db90b0570c7f410a78"}
Sep 11 13:23:32.864741 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:32.864723    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:23:32.865195 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:32.865172    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/1.log"
Sep 11 13:23:32.865274 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:32.865220    2580 generic.go:356] "Generic (PLEG): container finished" podID="9d5a874c-26f3-4238-91aa-70bb9c2938ae" containerID="e13bba567e0448ce5e67e8980c2d906dd41bbb26821345e67fe98d7fdf4be950" exitCode=255
Sep 11 13:23:32.865325 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:32.865287    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" event={"ID":"9d5a874c-26f3-4238-91aa-70bb9c2938ae","Type":"ContainerDied","Data":"e13bba567e0448ce5e67e8980c2d906dd41bbb26821345e67fe98d7fdf4be950"}
Sep 11 13:23:32.865375 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:32.865338    2580 scope.go:117] "RemoveContainer" containerID="e55a66546811367b30656e3dcb3d007c8c7d70a66cba9ad62ad42b69d6179c53"
Sep 11 13:23:32.865885 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:32.865633    2580 scope.go:117] "RemoveContainer" containerID="e13bba567e0448ce5e67e8980c2d906dd41bbb26821345e67fe98d7fdf4be950"
Sep 11 13:23:32.865885 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:32.865835    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-mjvgv_openshift-console-operator(9d5a874c-26f3-4238-91aa-70bb9c2938ae)\"" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" podUID="9d5a874c-26f3-4238-91aa-70bb9c2938ae"
Sep 11 13:23:33.620761 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:33.620724    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-5wm6g\" (UID: \"b76062ff-6ab0-4d3a-b139-12b777cb0ce6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:33.623059 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:33.623036    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b76062ff-6ab0-4d3a-b139-12b777cb0ce6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-5wm6g\" (UID: \"b76062ff-6ab0-4d3a-b139-12b777cb0ce6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:33.869892 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:33.869852    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hxs7m" event={"ID":"30dcd97e-d15b-4ca5-ac15-997c0ba69cf1","Type":"ContainerStarted","Data":"64e50fed274ee6822118d0e2932e82e99db91f4afe78f8c8d77d5e03683dde5e"}
Sep 11 13:23:33.871357 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:33.871298    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:23:33.887081 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:33.887036    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hxs7m" podStartSLOduration=131.707139166 podStartE2EDuration="2m12.887019882s" totalImagesPullingTime="1.179880716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:33.886684557 +0000 UTC m=+133.105222049" watchObservedRunningTime="2026-09-11 13:23:33.887019882 +0000 UTC m=+133.105557373"
Sep 11 13:23:33.901110 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:33.901087    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ldtdj\""
Sep 11 13:23:33.909338 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:33.909315    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"
Sep 11 13:23:34.179973 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:34.179739    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g"]
Sep 11 13:23:34.182423 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:23:34.182395    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76062ff_6ab0_4d3a_b139_12b777cb0ce6.slice/crio-307d7c119efacc38d303f4814a694e4e74d06c702be48f2b233fc30e2243aa3b WatchSource:0}: Error finding container 307d7c119efacc38d303f4814a694e4e74d06c702be48f2b233fc30e2243aa3b: Status 404 returned error can't find the container with id 307d7c119efacc38d303f4814a694e4e74d06c702be48f2b233fc30e2243aa3b
Sep 11 13:23:34.878516 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:34.878480    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g" event={"ID":"b76062ff-6ab0-4d3a-b139-12b777cb0ce6","Type":"ContainerStarted","Data":"307d7c119efacc38d303f4814a694e4e74d06c702be48f2b233fc30e2243aa3b"}
Sep 11 13:23:36.887259 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:36.887223    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g" event={"ID":"b76062ff-6ab0-4d3a-b139-12b777cb0ce6","Type":"ContainerStarted","Data":"dc6677928848fa607df57e0f03c11aaaf046e83ebf999c8e2cf717f59d8258ec"}
Sep 11 13:23:36.906169 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:36.906117    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-5wm6g" podStartSLOduration=34.132875528 podStartE2EDuration="35.906103126s" totalImagesPullingTime="1.773227598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:23:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:36.905695921 +0000 UTC m=+136.124233425" watchObservedRunningTime="2026-09-11 13:23:36.906103126 +0000 UTC m=+136.124640619"
Sep 11 13:23:37.025475 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:37.025439    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:37.025813 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:37.025795    2580 scope.go:117] "RemoveContainer" containerID="e13bba567e0448ce5e67e8980c2d906dd41bbb26821345e67fe98d7fdf4be950"
Sep 11 13:23:37.025987 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:37.025970    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-mjvgv_openshift-console-operator(9d5a874c-26f3-4238-91aa-70bb9c2938ae)\"" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" podUID="9d5a874c-26f3-4238-91aa-70bb9c2938ae"
Sep 11 13:23:37.850777 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:37.850726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:37.852949 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:37.852930    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09ffbcb6-3382-48ea-9c3a-04cf17da047e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-85bpc\" (UID: \"09ffbcb6-3382-48ea-9c3a-04cf17da047e\") " pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:38.115629 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:38.115546    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kd6rn\""
Sep 11 13:23:38.122553 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:38.122530    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-85bpc"
Sep 11 13:23:38.405611 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:38.405529    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-85bpc"]
Sep 11 13:23:38.409353 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:23:38.409323    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09ffbcb6_3382_48ea_9c3a_04cf17da047e.slice/crio-5bee44f33846cc42d9f5c0658ed91cc005fe7e1d91ac424a0e2a7d7eaa1eb2b9 WatchSource:0}: Error finding container 5bee44f33846cc42d9f5c0658ed91cc005fe7e1d91ac424a0e2a7d7eaa1eb2b9: Status 404 returned error can't find the container with id 5bee44f33846cc42d9f5c0658ed91cc005fe7e1d91ac424a0e2a7d7eaa1eb2b9
Sep 11 13:23:38.895920 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:38.895882    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-85bpc" event={"ID":"09ffbcb6-3382-48ea-9c3a-04cf17da047e","Type":"ContainerStarted","Data":"ab7247ce37d9060b0c5ee25b1755c8ba55b27ff0d025fe7ec7f3d5e5dfce7603"}
Sep 11 13:23:38.896094 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:38.895968    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-85bpc" event={"ID":"09ffbcb6-3382-48ea-9c3a-04cf17da047e","Type":"ContainerStarted","Data":"5bee44f33846cc42d9f5c0658ed91cc005fe7e1d91ac424a0e2a7d7eaa1eb2b9"}
Sep 11 13:23:39.754327 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:39.754292    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:23:39.754725 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:39.754709    2580 scope.go:117] "RemoveContainer" containerID="e13bba567e0448ce5e67e8980c2d906dd41bbb26821345e67fe98d7fdf4be950"
Sep 11 13:23:39.754879 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:39.754861    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-mjvgv_openshift-console-operator(9d5a874c-26f3-4238-91aa-70bb9c2938ae)\"" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" podUID="9d5a874c-26f3-4238-91aa-70bb9c2938ae"
Sep 11 13:23:39.902327 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:39.902289    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-85bpc" event={"ID":"09ffbcb6-3382-48ea-9c3a-04cf17da047e","Type":"ContainerStarted","Data":"feb125b7469c079e01de59cf804742dee71f4c949ebcd90410dcd0f4fa84289e"}
Sep 11 13:23:40.907385 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:40.907351    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-85bpc" event={"ID":"09ffbcb6-3382-48ea-9c3a-04cf17da047e","Type":"ContainerStarted","Data":"44852cf952be739250c36c14f3a428744a526dcc993a7f089bb3273ce5908feb"}
Sep 11 13:23:40.928106 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:40.928054    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-85bpc" podStartSLOduration=34.071975484 podStartE2EDuration="35.928037504s" totalImagesPullingTime="1.85606202s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:23:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:40.926643991 +0000 UTC m=+140.145181486" watchObservedRunningTime="2026-09-11 13:23:40.928037504 +0000 UTC m=+140.146574987"
Sep 11 13:23:42.779844 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.779806    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-tn4r8"]
Sep 11 13:23:42.782778 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.782758    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tn4r8"
Sep 11 13:23:42.786458 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.786436    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-z56jp\""
Sep 11 13:23:42.786563 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.786464    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 11 13:23:42.786563 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.786537    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 11 13:23:42.796716 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.796691    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-tn4r8"]
Sep 11 13:23:42.869337 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.869302    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7db97697d-ndhf8"]
Sep 11 13:23:42.872408 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.872384    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:42.876528 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.876503    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zq2ct\""
Sep 11 13:23:42.877184 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.877165    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 13:23:42.878065 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.878045    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 13:23:42.882384 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.882368    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 13:23:42.888061 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.888046    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 13:23:42.889997 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.889981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/11bc16f7-1763-49ec-ae74-2783e7970924-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-tn4r8\" (UID: \"11bc16f7-1763-49ec-ae74-2783e7970924\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tn4r8"
Sep 11 13:23:42.890045 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.890011    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/11bc16f7-1763-49ec-ae74-2783e7970924-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-tn4r8\" (UID: \"11bc16f7-1763-49ec-ae74-2783e7970924\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tn4r8"
Sep 11 13:23:42.897103 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.897073    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7db97697d-ndhf8"]
Sep 11 13:23:42.990890 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.990859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/83408741-3754-4200-9226-2d58ea07355c-installation-pull-secrets\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:42.990890 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.990893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/83408741-3754-4200-9226-2d58ea07355c-ca-trust-extracted\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:42.991095 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.990967    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/83408741-3754-4200-9226-2d58ea07355c-image-registry-private-configuration\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:42.991095 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.991003    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/83408741-3754-4200-9226-2d58ea07355c-trusted-ca\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:42.991095 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.991043    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83408741-3754-4200-9226-2d58ea07355c-registry-tls\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:42.991095 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.991065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83408741-3754-4200-9226-2d58ea07355c-bound-sa-token\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:42.991095 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.991086    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfg8p\" (UniqueName: \"kubernetes.io/projected/83408741-3754-4200-9226-2d58ea07355c-kube-api-access-gfg8p\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:42.991239 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.991124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/11bc16f7-1763-49ec-ae74-2783e7970924-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-tn4r8\" (UID: \"11bc16f7-1763-49ec-ae74-2783e7970924\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tn4r8"
Sep 11 13:23:42.991239 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.991151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/11bc16f7-1763-49ec-ae74-2783e7970924-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-tn4r8\" (UID: \"11bc16f7-1763-49ec-ae74-2783e7970924\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tn4r8"
Sep 11 13:23:42.991239 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.991183    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/83408741-3754-4200-9226-2d58ea07355c-registry-certificates\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:42.991777 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.991756    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/11bc16f7-1763-49ec-ae74-2783e7970924-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-tn4r8\" (UID: \"11bc16f7-1763-49ec-ae74-2783e7970924\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tn4r8"
Sep 11 13:23:42.993451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:42.993430    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/11bc16f7-1763-49ec-ae74-2783e7970924-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-tn4r8\" (UID: \"11bc16f7-1763-49ec-ae74-2783e7970924\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tn4r8"
Sep 11 13:23:43.091838 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.091807    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tn4r8"
Sep 11 13:23:43.092039 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.092020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83408741-3754-4200-9226-2d58ea07355c-bound-sa-token\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.092113 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.092058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfg8p\" (UniqueName: \"kubernetes.io/projected/83408741-3754-4200-9226-2d58ea07355c-kube-api-access-gfg8p\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.092174 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.092126    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/83408741-3754-4200-9226-2d58ea07355c-registry-certificates\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.092174 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.092152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/83408741-3754-4200-9226-2d58ea07355c-installation-pull-secrets\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.092266 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.092180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/83408741-3754-4200-9226-2d58ea07355c-ca-trust-extracted\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.092266 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.092221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/83408741-3754-4200-9226-2d58ea07355c-image-registry-private-configuration\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.092266 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.092248    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/83408741-3754-4200-9226-2d58ea07355c-trusted-ca\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.092425 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.092285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83408741-3754-4200-9226-2d58ea07355c-registry-tls\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.092688 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.092662    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/83408741-3754-4200-9226-2d58ea07355c-ca-trust-extracted\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.093117 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.093095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/83408741-3754-4200-9226-2d58ea07355c-registry-certificates\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.093386 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.093361    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/83408741-3754-4200-9226-2d58ea07355c-trusted-ca\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.094813 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.094792    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/83408741-3754-4200-9226-2d58ea07355c-installation-pull-secrets\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.094948 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.094875    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83408741-3754-4200-9226-2d58ea07355c-registry-tls\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.095043 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.095024    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/83408741-3754-4200-9226-2d58ea07355c-image-registry-private-configuration\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.100593 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.100566    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83408741-3754-4200-9226-2d58ea07355c-bound-sa-token\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.100823 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.100806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfg8p\" (UniqueName: \"kubernetes.io/projected/83408741-3754-4200-9226-2d58ea07355c-kube-api-access-gfg8p\") pod \"image-registry-7db97697d-ndhf8\" (UID: \"83408741-3754-4200-9226-2d58ea07355c\") " pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.181965 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.181927    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.364736 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.364713    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-tn4r8"]
Sep 11 13:23:43.366737 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:23:43.366705    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11bc16f7_1763_49ec_ae74_2783e7970924.slice/crio-c7395f8a2179e210c2a6483aff6888636365f76859d051ce90297a410c5ac8de WatchSource:0}: Error finding container c7395f8a2179e210c2a6483aff6888636365f76859d051ce90297a410c5ac8de: Status 404 returned error can't find the container with id c7395f8a2179e210c2a6483aff6888636365f76859d051ce90297a410c5ac8de
Sep 11 13:23:43.459616 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.459587    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7db97697d-ndhf8"]
Sep 11 13:23:43.461763 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:23:43.461735    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83408741_3754_4200_9226_2d58ea07355c.slice/crio-b05c9668eafe192adaee38237a07fbb5c9d7bc975d9862c0eb9d929531ce657d WatchSource:0}: Error finding container b05c9668eafe192adaee38237a07fbb5c9d7bc975d9862c0eb9d929531ce657d: Status 404 returned error can't find the container with id b05c9668eafe192adaee38237a07fbb5c9d7bc975d9862c0eb9d929531ce657d
Sep 11 13:23:43.920950 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.920912    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tn4r8" event={"ID":"11bc16f7-1763-49ec-ae74-2783e7970924","Type":"ContainerStarted","Data":"c7395f8a2179e210c2a6483aff6888636365f76859d051ce90297a410c5ac8de"}
Sep 11 13:23:43.922575 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.922549    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7db97697d-ndhf8" event={"ID":"83408741-3754-4200-9226-2d58ea07355c","Type":"ContainerStarted","Data":"90c73b7cc291424cc7b928c1c6a4301a31dde37eeb3c037b09c4f1d0c434a032"}
Sep 11 13:23:43.922706 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.922592    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7db97697d-ndhf8" event={"ID":"83408741-3754-4200-9226-2d58ea07355c","Type":"ContainerStarted","Data":"b05c9668eafe192adaee38237a07fbb5c9d7bc975d9862c0eb9d929531ce657d"}
Sep 11 13:23:43.922762 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.922714    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:23:43.942734 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:43.942688    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7db97697d-ndhf8" podStartSLOduration=1.942673105 podStartE2EDuration="1.942673105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:23:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:43.942221784 +0000 UTC m=+143.160759277" watchObservedRunningTime="2026-09-11 13:23:43.942673105 +0000 UTC m=+143.161210593"
Sep 11 13:23:44.927592 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:44.927506    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tn4r8" event={"ID":"11bc16f7-1763-49ec-ae74-2783e7970924","Type":"ContainerStarted","Data":"f2650d5a6d9a4b48f222ea2ece406566259286710649525ab2a7f8218bc092a5"}
Sep 11 13:23:44.944433 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:44.944372    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tn4r8" podStartSLOduration=1.6533877449999999 podStartE2EDuration="2.944350656s" totalImagesPullingTime="1.290962911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:23:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:44.944207221 +0000 UTC m=+144.162744714" watchObservedRunningTime="2026-09-11 13:23:44.944350656 +0000 UTC m=+144.162888148"
Sep 11 13:23:49.991322 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:49.991277    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q2lnk"]
Sep 11 13:23:49.996463 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:49.996437    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:49.999098 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:49.999064    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 13:23:49.999551 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:49.999388    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 13:23:49.999672 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:49.999607    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 13:23:49.999738 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:49.999674    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pmtd5\""
Sep 11 13:23:50.000419 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.000397    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 13:23:50.049234 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.049199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44626f2f-a336-45c7-8623-31f5f7ac93ca-sys\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.049412 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.049249    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44626f2f-a336-45c7-8623-31f5f7ac93ca-metrics-client-ca\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.049412 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.049286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/44626f2f-a336-45c7-8623-31f5f7ac93ca-node-exporter-accelerators-collector-config\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.049412 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.049365    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44626f2f-a336-45c7-8623-31f5f7ac93ca-node-exporter-tls\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.049588 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.049423    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44626f2f-a336-45c7-8623-31f5f7ac93ca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.049588 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.049469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/44626f2f-a336-45c7-8623-31f5f7ac93ca-node-exporter-wtmp\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.049588 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.049508    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/44626f2f-a336-45c7-8623-31f5f7ac93ca-root\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.049588 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.049536    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/44626f2f-a336-45c7-8623-31f5f7ac93ca-node-exporter-textfile\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.049588 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.049563    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2lc5\" (UniqueName: \"kubernetes.io/projected/44626f2f-a336-45c7-8623-31f5f7ac93ca-kube-api-access-d2lc5\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.150675 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.150642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/44626f2f-a336-45c7-8623-31f5f7ac93ca-node-exporter-accelerators-collector-config\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.150833 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.150692    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44626f2f-a336-45c7-8623-31f5f7ac93ca-node-exporter-tls\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.150833 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.150723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44626f2f-a336-45c7-8623-31f5f7ac93ca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.150833 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.150742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/44626f2f-a336-45c7-8623-31f5f7ac93ca-node-exporter-wtmp\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.150833 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.150815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/44626f2f-a336-45c7-8623-31f5f7ac93ca-root\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.151096 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.150856    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/44626f2f-a336-45c7-8623-31f5f7ac93ca-node-exporter-textfile\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.151096 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.150883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2lc5\" (UniqueName: \"kubernetes.io/projected/44626f2f-a336-45c7-8623-31f5f7ac93ca-kube-api-access-d2lc5\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.151096 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.150893    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/44626f2f-a336-45c7-8623-31f5f7ac93ca-node-exporter-wtmp\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.151096 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.150966    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44626f2f-a336-45c7-8623-31f5f7ac93ca-sys\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.151096 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.151003    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44626f2f-a336-45c7-8623-31f5f7ac93ca-metrics-client-ca\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.151372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.151346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/44626f2f-a336-45c7-8623-31f5f7ac93ca-node-exporter-accelerators-collector-config\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.151372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.151356    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/44626f2f-a336-45c7-8623-31f5f7ac93ca-node-exporter-textfile\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.151483 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.151431    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44626f2f-a336-45c7-8623-31f5f7ac93ca-sys\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.151570 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.151553    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/44626f2f-a336-45c7-8623-31f5f7ac93ca-root\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.151622 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.151555    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44626f2f-a336-45c7-8623-31f5f7ac93ca-metrics-client-ca\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.153199 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.153177    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44626f2f-a336-45c7-8623-31f5f7ac93ca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.153554 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.153534    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44626f2f-a336-45c7-8623-31f5f7ac93ca-node-exporter-tls\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.161243 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.161220    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2lc5\" (UniqueName: \"kubernetes.io/projected/44626f2f-a336-45c7-8623-31f5f7ac93ca-kube-api-access-d2lc5\") pod \"node-exporter-q2lnk\" (UID: \"44626f2f-a336-45c7-8623-31f5f7ac93ca\") " pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.307160 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.307076    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q2lnk"
Sep 11 13:23:50.318761 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:23:50.318731    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44626f2f_a336_45c7_8623_31f5f7ac93ca.slice/crio-91ac0098bc7e8c3e07a4302a229f24d2f8c6c8e4348a18c3536cc732fd836b90 WatchSource:0}: Error finding container 91ac0098bc7e8c3e07a4302a229f24d2f8c6c8e4348a18c3536cc732fd836b90: Status 404 returned error can't find the container with id 91ac0098bc7e8c3e07a4302a229f24d2f8c6c8e4348a18c3536cc732fd836b90
Sep 11 13:23:50.950752 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:50.950715    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q2lnk" event={"ID":"44626f2f-a336-45c7-8623-31f5f7ac93ca","Type":"ContainerStarted","Data":"91ac0098bc7e8c3e07a4302a229f24d2f8c6c8e4348a18c3536cc732fd836b90"}
Sep 11 13:23:51.311297 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:51.311263    2580 scope.go:117] "RemoveContainer" containerID="e13bba567e0448ce5e67e8980c2d906dd41bbb26821345e67fe98d7fdf4be950"
Sep 11 13:23:51.311722 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:51.311465    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-mjvgv_openshift-console-operator(9d5a874c-26f3-4238-91aa-70bb9c2938ae)\"" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" podUID="9d5a874c-26f3-4238-91aa-70bb9c2938ae"
Sep 11 13:23:51.955307 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:51.955271    2580 generic.go:356] "Generic (PLEG): container finished" podID="44626f2f-a336-45c7-8623-31f5f7ac93ca" containerID="d0ef8fbf1a772be6f35c89a0191cadee2efa39ddb5873bc76ff90069248ac4c8" exitCode=0
Sep 11 13:23:51.955473 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:51.955343    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q2lnk" event={"ID":"44626f2f-a336-45c7-8623-31f5f7ac93ca","Type":"ContainerDied","Data":"d0ef8fbf1a772be6f35c89a0191cadee2efa39ddb5873bc76ff90069248ac4c8"}
Sep 11 13:23:52.961809 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:52.961770    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q2lnk" event={"ID":"44626f2f-a336-45c7-8623-31f5f7ac93ca","Type":"ContainerStarted","Data":"f5cfae816403fd3eead299656024632ad475586744bc990bad548273a0cb15e7"}
Sep 11 13:23:52.962187 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:52.961821    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q2lnk" event={"ID":"44626f2f-a336-45c7-8623-31f5f7ac93ca","Type":"ContainerStarted","Data":"0915ddc478ea3b2669d34e787a973816ce0a2553d99303553f046cab2a8bb747"}
Sep 11 13:23:53.026623 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:53.026563    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q2lnk" podStartSLOduration=3.058062639 podStartE2EDuration="4.026544062s" totalImagesPullingTime="904.212978ms" totalInitContainerRuntime="64.268445ms" isStatefulPod=true podCreationTimestamp="2026-09-11 13:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:53.018219262 +0000 UTC m=+152.236756753" watchObservedRunningTime="2026-09-11 13:23:53.026544062 +0000 UTC m=+152.245081555"
Sep 11 13:23:54.720519 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:54.720484    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj"]
Sep 11 13:23:54.724071 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:54.724052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj"
Sep 11 13:23:54.729079 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:54.729055    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 11 13:23:54.729252 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:54.729234    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-lpjds\""
Sep 11 13:23:54.738582 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:54.738560    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj"]
Sep 11 13:23:54.789224 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:54.789190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gtq7\" (UniqueName: \"kubernetes.io/projected/4ca70175-727d-4083-859c-095e0d48e811-kube-api-access-8gtq7\") pod \"monitoring-plugin-5d85778b9d-zkgsj\" (UID: \"4ca70175-727d-4083-859c-095e0d48e811\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj"
Sep 11 13:23:54.789384 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:54.789245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ca70175-727d-4083-859c-095e0d48e811-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-zkgsj\" (UID: \"4ca70175-727d-4083-859c-095e0d48e811\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj"
Sep 11 13:23:54.890432 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:54.890400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gtq7\" (UniqueName: \"kubernetes.io/projected/4ca70175-727d-4083-859c-095e0d48e811-kube-api-access-8gtq7\") pod \"monitoring-plugin-5d85778b9d-zkgsj\" (UID: \"4ca70175-727d-4083-859c-095e0d48e811\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj"
Sep 11 13:23:54.890575 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:54.890463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ca70175-727d-4083-859c-095e0d48e811-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-zkgsj\" (UID: \"4ca70175-727d-4083-859c-095e0d48e811\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj"
Sep 11 13:23:54.890617 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:54.890583    2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 11 13:23:54.890647 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:54.890639    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ca70175-727d-4083-859c-095e0d48e811-monitoring-plugin-cert podName:4ca70175-727d-4083-859c-095e0d48e811 nodeName:}" failed. No retries permitted until 2026-09-11 13:23:55.390622609 +0000 UTC m=+154.609160078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/4ca70175-727d-4083-859c-095e0d48e811-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-zkgsj" (UID: "4ca70175-727d-4083-859c-095e0d48e811") : secret "monitoring-plugin-cert" not found
Sep 11 13:23:54.901839 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:54.901804    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gtq7\" (UniqueName: \"kubernetes.io/projected/4ca70175-727d-4083-859c-095e0d48e811-kube-api-access-8gtq7\") pod \"monitoring-plugin-5d85778b9d-zkgsj\" (UID: \"4ca70175-727d-4083-859c-095e0d48e811\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj"
Sep 11 13:23:55.196163 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.196125    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5cc665f996-jzwh2"]
Sep 11 13:23:55.199371 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.199353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.202220 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.202186    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-hvc2l\""
Sep 11 13:23:55.203068 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.203047    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 11 13:23:55.203179 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.203140    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 11 13:23:55.203179 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.203141    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 11 13:23:55.203533 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.203513    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 11 13:23:55.204039 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.204021    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 11 13:23:55.208551 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.208535    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 11 13:23:55.237478 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.237451    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5cc665f996-jzwh2"]
Sep 11 13:23:55.293808 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.293761    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43e0b77a-f94e-4b10-b701-81445648a729-serving-certs-ca-bundle\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.294003 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.293836    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/43e0b77a-f94e-4b10-b701-81445648a729-federate-client-tls\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.294003 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.293866    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43e0b77a-f94e-4b10-b701-81445648a729-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.294003 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.293987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43e0b77a-f94e-4b10-b701-81445648a729-metrics-client-ca\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.294141 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.294027    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43e0b77a-f94e-4b10-b701-81445648a729-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.294141 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.294049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/43e0b77a-f94e-4b10-b701-81445648a729-secret-telemeter-client\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.294141 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.294071    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/43e0b77a-f94e-4b10-b701-81445648a729-telemeter-client-tls\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.294141 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.294098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dhmb\" (UniqueName: \"kubernetes.io/projected/43e0b77a-f94e-4b10-b701-81445648a729-kube-api-access-2dhmb\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.394598 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.394561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43e0b77a-f94e-4b10-b701-81445648a729-serving-certs-ca-bundle\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.394802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.394610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/43e0b77a-f94e-4b10-b701-81445648a729-federate-client-tls\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.394802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.394632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43e0b77a-f94e-4b10-b701-81445648a729-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.394802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.394666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ca70175-727d-4083-859c-095e0d48e811-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-zkgsj\" (UID: \"4ca70175-727d-4083-859c-095e0d48e811\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj"
Sep 11 13:23:55.394802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.394683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43e0b77a-f94e-4b10-b701-81445648a729-metrics-client-ca\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.394802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.394700    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43e0b77a-f94e-4b10-b701-81445648a729-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.394802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.394717    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/43e0b77a-f94e-4b10-b701-81445648a729-secret-telemeter-client\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.394802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.394735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/43e0b77a-f94e-4b10-b701-81445648a729-telemeter-client-tls\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.394802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.394792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dhmb\" (UniqueName: \"kubernetes.io/projected/43e0b77a-f94e-4b10-b701-81445648a729-kube-api-access-2dhmb\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.395457 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.395432    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43e0b77a-f94e-4b10-b701-81445648a729-serving-certs-ca-bundle\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.395776 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.395712    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43e0b77a-f94e-4b10-b701-81445648a729-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.395959 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.395939    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43e0b77a-f94e-4b10-b701-81445648a729-metrics-client-ca\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.397515 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.397485    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/43e0b77a-f94e-4b10-b701-81445648a729-telemeter-client-tls\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.397879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.397860    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4ca70175-727d-4083-859c-095e0d48e811-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-zkgsj\" (UID: \"4ca70175-727d-4083-859c-095e0d48e811\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj"
Sep 11 13:23:55.397941 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.397863    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/43e0b77a-f94e-4b10-b701-81445648a729-secret-telemeter-client\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.397983 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.397940    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43e0b77a-f94e-4b10-b701-81445648a729-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.397983 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.397967    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/43e0b77a-f94e-4b10-b701-81445648a729-federate-client-tls\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.403430 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.403410    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dhmb\" (UniqueName: \"kubernetes.io/projected/43e0b77a-f94e-4b10-b701-81445648a729-kube-api-access-2dhmb\") pod \"telemeter-client-5cc665f996-jzwh2\" (UID: \"43e0b77a-f94e-4b10-b701-81445648a729\") " pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.508841 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.508758    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2"
Sep 11 13:23:55.632282 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.632241    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj"
Sep 11 13:23:55.790555 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.790532    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5cc665f996-jzwh2"]
Sep 11 13:23:55.792456 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:23:55.792430    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43e0b77a_f94e_4b10_b701_81445648a729.slice/crio-5ced8fbdbaefa6a2e54ff811fd52855e77f54c12826a8a84f20253c76aabdc59 WatchSource:0}: Error finding container 5ced8fbdbaefa6a2e54ff811fd52855e77f54c12826a8a84f20253c76aabdc59: Status 404 returned error can't find the container with id 5ced8fbdbaefa6a2e54ff811fd52855e77f54c12826a8a84f20253c76aabdc59
Sep 11 13:23:55.905775 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.905748    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj"]
Sep 11 13:23:55.909024 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:23:55.908994    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca70175_727d_4083_859c_095e0d48e811.slice/crio-4a6e4a61e33fda33d4f0c1667ffe1a764dbc8ee21f2a0b58c3260daa74372f73 WatchSource:0}: Error finding container 4a6e4a61e33fda33d4f0c1667ffe1a764dbc8ee21f2a0b58c3260daa74372f73: Status 404 returned error can't find the container with id 4a6e4a61e33fda33d4f0c1667ffe1a764dbc8ee21f2a0b58c3260daa74372f73
Sep 11 13:23:55.972727 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.972698    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2" event={"ID":"43e0b77a-f94e-4b10-b701-81445648a729","Type":"ContainerStarted","Data":"5ced8fbdbaefa6a2e54ff811fd52855e77f54c12826a8a84f20253c76aabdc59"}
Sep 11 13:23:55.973835 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:55.973811    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj" event={"ID":"4ca70175-727d-4083-859c-095e0d48e811","Type":"ContainerStarted","Data":"4a6e4a61e33fda33d4f0c1667ffe1a764dbc8ee21f2a0b58c3260daa74372f73"}
Sep 11 13:23:57.984370 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:57.984275    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj" event={"ID":"4ca70175-727d-4083-859c-095e0d48e811","Type":"ContainerStarted","Data":"638a08c60cd2a373b552fc547e7cff62c0b6d541ae6da51eb80637d643e465fa"}
Sep 11 13:23:57.984818 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:57.984425    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj"
Sep 11 13:23:57.986003 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:57.985969    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2" event={"ID":"43e0b77a-f94e-4b10-b701-81445648a729","Type":"ContainerStarted","Data":"a22b52ce276364d5759a7421bd1f65f7a25611e15e93cc1794da27e7314a7685"}
Sep 11 13:23:57.989325 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:57.989301    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj"
Sep 11 13:23:58.003115 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:58.003070    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-zkgsj" podStartSLOduration=2.2546146289999998 podStartE2EDuration="4.0030571s" totalImagesPullingTime="1.748442471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:58.002283302 +0000 UTC m=+157.220820790" watchObservedRunningTime="2026-09-11 13:23:58.0030571 +0000 UTC m=+157.221594592"
Sep 11 13:23:58.603844 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:23:58.603800    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9wxh8" podUID="934019ff-93c3-464b-906f-39c185f6d5e4"
Sep 11 13:23:58.992373 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:58.992289    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:23:58.992779 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:58.992283    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2" event={"ID":"43e0b77a-f94e-4b10-b701-81445648a729","Type":"ContainerStarted","Data":"fd75152c961c478541c7743d7a3f4a4997a9e66f1fd48c6a7d67c7116c2126a0"}
Sep 11 13:23:58.992779 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:58.992433    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2" event={"ID":"43e0b77a-f94e-4b10-b701-81445648a729","Type":"ContainerStarted","Data":"9bb7ddeb9915746fca224acb65f2c599510c44cb3bfad92cebd493c9fedbbea5"}
Sep 11 13:23:59.032915 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:23:59.032864    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5cc665f996-jzwh2" podStartSLOduration=1.2162011480000001 podStartE2EDuration="4.032850228s" totalImagesPullingTime="2.81664908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:23:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:59.032365818 +0000 UTC m=+158.250903310" watchObservedRunningTime="2026-09-11 13:23:59.032850228 +0000 UTC m=+158.251387720"
Sep 11 13:24:03.186848 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:03.186803    2580 patch_prober.go:28] interesting pod/image-registry-7db97697d-ndhf8 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 11 13:24:03.187325 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:03.186862    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7db97697d-ndhf8" podUID="83408741-3754-4200-9226-2d58ea07355c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:24:03.464533 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:03.464420    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:24:03.466723 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:03.466698    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/934019ff-93c3-464b-906f-39c185f6d5e4-metrics-tls\") pod \"dns-default-9wxh8\" (UID: \"934019ff-93c3-464b-906f-39c185f6d5e4\") " pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:24:03.495567 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:03.495539    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vqfbt\""
Sep 11 13:24:03.503011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:03.502992    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:24:03.773606 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:03.773547    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9wxh8"]
Sep 11 13:24:03.776134 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:24:03.776107    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod934019ff_93c3_464b_906f_39c185f6d5e4.slice/crio-d831101cac9dc4c3ac956d7eaa7f570d59367ab738a6013c6d765d2beb864b3e WatchSource:0}: Error finding container d831101cac9dc4c3ac956d7eaa7f570d59367ab738a6013c6d765d2beb864b3e: Status 404 returned error can't find the container with id d831101cac9dc4c3ac956d7eaa7f570d59367ab738a6013c6d765d2beb864b3e
Sep 11 13:24:04.014696 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:04.014653    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9wxh8" event={"ID":"934019ff-93c3-464b-906f-39c185f6d5e4","Type":"ContainerStarted","Data":"d831101cac9dc4c3ac956d7eaa7f570d59367ab738a6013c6d765d2beb864b3e"}
Sep 11 13:24:04.309919 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:04.309887    2580 scope.go:117] "RemoveContainer" containerID="e13bba567e0448ce5e67e8980c2d906dd41bbb26821345e67fe98d7fdf4be950"
Sep 11 13:24:04.933086 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:04.933036    2580 patch_prober.go:28] interesting pod/image-registry-7db97697d-ndhf8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 11 13:24:04.933281 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:04.933115    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7db97697d-ndhf8" podUID="83408741-3754-4200-9226-2d58ea07355c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:24:05.025633 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:05.025601    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:24:05.025802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:05.025703    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv" event={"ID":"9d5a874c-26f3-4238-91aa-70bb9c2938ae","Type":"ContainerStarted","Data":"110812e9f752ef44d6664a9ac47281894f79d6a3cfe36c95225c66fa65336385"}
Sep 11 13:24:05.026014 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:05.025981    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:24:05.030780 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:05.030755    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-mjvgv"
Sep 11 13:24:05.078946 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:05.078894    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-4h24k"]
Sep 11 13:24:05.081368 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:05.081349    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-4h24k"
Sep 11 13:24:05.084047 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:05.084007    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 13:24:05.084365 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:05.084009    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 13:24:05.084365 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:05.084332    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-bh998\""
Sep 11 13:24:05.100227 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:05.100204    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-4h24k"]
Sep 11 13:24:05.178190 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:05.178161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95j78\" (UniqueName: \"kubernetes.io/projected/efcfccc2-da3a-41eb-96c7-31114c25ad5d-kube-api-access-95j78\") pod \"downloads-76f99584f4-4h24k\" (UID: \"efcfccc2-da3a-41eb-96c7-31114c25ad5d\") " pod="openshift-console/downloads-76f99584f4-4h24k"
Sep 11 13:24:05.279285 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:05.279193    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95j78\" (UniqueName: \"kubernetes.io/projected/efcfccc2-da3a-41eb-96c7-31114c25ad5d-kube-api-access-95j78\") pod \"downloads-76f99584f4-4h24k\" (UID: \"efcfccc2-da3a-41eb-96c7-31114c25ad5d\") " pod="openshift-console/downloads-76f99584f4-4h24k"
Sep 11 13:24:05.289472 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:05.289447    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95j78\" (UniqueName: \"kubernetes.io/projected/efcfccc2-da3a-41eb-96c7-31114c25ad5d-kube-api-access-95j78\") pod \"downloads-76f99584f4-4h24k\" (UID: \"efcfccc2-da3a-41eb-96c7-31114c25ad5d\") " pod="openshift-console/downloads-76f99584f4-4h24k"
Sep 11 13:24:05.392519 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:05.392475    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-4h24k"
Sep 11 13:24:05.769632 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:05.769602    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-4h24k"]
Sep 11 13:24:05.772319 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:24:05.772295    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefcfccc2_da3a_41eb_96c7_31114c25ad5d.slice/crio-ce02bd29ae229aad44fb1d94582823723e9d8d6b6dc6adb186c961d16b160dc1 WatchSource:0}: Error finding container ce02bd29ae229aad44fb1d94582823723e9d8d6b6dc6adb186c961d16b160dc1: Status 404 returned error can't find the container with id ce02bd29ae229aad44fb1d94582823723e9d8d6b6dc6adb186c961d16b160dc1
Sep 11 13:24:06.031481 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:06.031446    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9wxh8" event={"ID":"934019ff-93c3-464b-906f-39c185f6d5e4","Type":"ContainerStarted","Data":"b3b434eee5828810fb4b154340f7f40c17decc33eb34db7bdf01e06357a77a23"}
Sep 11 13:24:06.031661 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:06.031505    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9wxh8" event={"ID":"934019ff-93c3-464b-906f-39c185f6d5e4","Type":"ContainerStarted","Data":"3f316b1e2cf4ea15a9f2e9e250247c5bbb448ca7a0b44dcfab81fc9394456a94"}
Sep 11 13:24:06.031661 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:06.031539    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:24:06.032706 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:06.032681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-4h24k" event={"ID":"efcfccc2-da3a-41eb-96c7-31114c25ad5d","Type":"ContainerStarted","Data":"ce02bd29ae229aad44fb1d94582823723e9d8d6b6dc6adb186c961d16b160dc1"}
Sep 11 13:24:06.056619 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:06.056546    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9wxh8" podStartSLOduration=129.316233753 podStartE2EDuration="2m11.056530032s" totalImagesPullingTime="1.740296279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:06.054589503 +0000 UTC m=+165.273126995" watchObservedRunningTime="2026-09-11 13:24:06.056530032 +0000 UTC m=+165.275067527"
Sep 11 13:24:11.014293 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.014247    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f5447f885-msfnc"]
Sep 11 13:24:11.020669 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.020637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.026288 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.026263    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pzs67\""
Sep 11 13:24:11.026416 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.026316    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 13:24:11.026416 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.026340    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 13:24:11.026836 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.026817    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 13:24:11.026936 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.026816    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 13:24:11.026936 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.026877    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 13:24:11.034456 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.034408    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f5447f885-msfnc"]
Sep 11 13:24:11.134633 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.134590    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-oauth-serving-cert\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.134633 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.134636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-service-ca\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.134874 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.134672    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-oauth-config\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.134874 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.134734    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-config\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.134874 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.134753    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-serving-cert\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.134874 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.134831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhw86\" (UniqueName: \"kubernetes.io/projected/d5c871e9-28ba-47fc-9276-eba7713fd04b-kube-api-access-lhw86\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.235424 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.235386    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhw86\" (UniqueName: \"kubernetes.io/projected/d5c871e9-28ba-47fc-9276-eba7713fd04b-kube-api-access-lhw86\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.235611 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.235438    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-oauth-serving-cert\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.235611 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.235462    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-service-ca\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.235611 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.235509    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-oauth-config\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.235611 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.235550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-config\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.235611 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.235573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-serving-cert\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.236324 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.236287    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-oauth-serving-cert\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.236324 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.236319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-config\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.236499 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.236294    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-service-ca\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.238389 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.238367    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-serving-cert\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.238493 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.238371    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-oauth-config\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.248863 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.248834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhw86\" (UniqueName: \"kubernetes.io/projected/d5c871e9-28ba-47fc-9276-eba7713fd04b-kube-api-access-lhw86\") pod \"console-7f5447f885-msfnc\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") " pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.263349 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.263325    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9wxh8_934019ff-93c3-464b-906f-39c185f6d5e4/dns/0.log"
Sep 11 13:24:11.332788 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.332754    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:11.464919 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.464868    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9wxh8_934019ff-93c3-464b-906f-39c185f6d5e4/kube-rbac-proxy/0.log"
Sep 11 13:24:11.618312 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:11.618212    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f5447f885-msfnc"]
Sep 11 13:24:11.621480 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:24:11.621449    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5c871e9_28ba_47fc_9276_eba7713fd04b.slice/crio-a06082dd6484144b656006a89f1f3597337f8eeb0bbdf49c566aec4a88245ce8 WatchSource:0}: Error finding container a06082dd6484144b656006a89f1f3597337f8eeb0bbdf49c566aec4a88245ce8: Status 404 returned error can't find the container with id a06082dd6484144b656006a89f1f3597337f8eeb0bbdf49c566aec4a88245ce8
Sep 11 13:24:12.063112 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:12.063078    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5447f885-msfnc" event={"ID":"d5c871e9-28ba-47fc-9276-eba7713fd04b","Type":"ContainerStarted","Data":"a06082dd6484144b656006a89f1f3597337f8eeb0bbdf49c566aec4a88245ce8"}
Sep 11 13:24:12.465480 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:12.465378    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rkrhj_030910bd-d1c8-48c5-bc44-5e561b3fbf60/dns-node-resolver/0.log"
Sep 11 13:24:13.187476 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:13.187277    2580 patch_prober.go:28] interesting pod/image-registry-7db97697d-ndhf8 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 11 13:24:13.187476 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:13.187443    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7db97697d-ndhf8" podUID="83408741-3754-4200-9226-2d58ea07355c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:24:13.261954 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:13.261920    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7db97697d-ndhf8_83408741-3754-4200-9226-2d58ea07355c/registry/0.log"
Sep 11 13:24:13.661636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:13.661588    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gz6tv_c3b3d505-6502-4cc5-9aec-aef4cd996049/node-ca/0.log"
Sep 11 13:24:14.262458 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:14.262382    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5zw59_fc1353d3-7bc6-45bd-9ecf-c325b4469fb8/serve-healthcheck-canary/0.log"
Sep 11 13:24:14.932513 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:14.932423    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7db97697d-ndhf8"
Sep 11 13:24:15.079313 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:15.079265    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5447f885-msfnc" event={"ID":"d5c871e9-28ba-47fc-9276-eba7713fd04b","Type":"ContainerStarted","Data":"e61e2b913826110bddecd6c057a56bdd578070bbb19f36150a58ccee9d9d53ff"}
Sep 11 13:24:15.100672 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:15.100582    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f5447f885-msfnc" podStartSLOduration=2.051495257 podStartE2EDuration="5.100558391s" totalImagesPullingTime="3.049063134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:15.099435933 +0000 UTC m=+174.317973426" watchObservedRunningTime="2026-09-11 13:24:15.100558391 +0000 UTC m=+174.319095885"
Sep 11 13:24:16.039527 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:16.039498    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9wxh8"
Sep 11 13:24:21.333307 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.333269    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:21.333862 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.333331    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:21.339699 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.339673    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:21.365948 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.365898    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56978c4d8d-m8lnd"]
Sep 11 13:24:21.370587 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.370558    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.379160 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.379136    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 13:24:21.380389 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.380363    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56978c4d8d-m8lnd"]
Sep 11 13:24:21.428892 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.428853    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-oauth-config\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.429089 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.428894    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-oauth-serving-cert\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.429089 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.428944    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj2cz\" (UniqueName: \"kubernetes.io/projected/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-kube-api-access-gj2cz\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.429089 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.428969    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-serving-cert\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.429269 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.429185    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-config\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.429399 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.429374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-service-ca\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.429460 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.429410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-trusted-ca-bundle\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.529877 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.529838    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-service-ca\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.529877 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.529874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-trusted-ca-bundle\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.530127 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.529945    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-oauth-config\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.530127 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.529970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-oauth-serving-cert\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.530127 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.529992    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj2cz\" (UniqueName: \"kubernetes.io/projected/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-kube-api-access-gj2cz\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.530127 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.530023    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-serving-cert\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.530332 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.530192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-config\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.530747 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.530688    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-service-ca\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.530886 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.530772    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-oauth-serving-cert\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.530886 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.530789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-trusted-ca-bundle\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.531063 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.530893    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-config\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.533235 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.533210    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-oauth-config\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.533235 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.533221    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-serving-cert\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.539705 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.539685    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj2cz\" (UniqueName: \"kubernetes.io/projected/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-kube-api-access-gj2cz\") pod \"console-56978c4d8d-m8lnd\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") " pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:21.682483 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:21.682405    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:22.062236 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:22.062198    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56978c4d8d-m8lnd"]
Sep 11 13:24:22.066982 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:24:22.066952    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73bf35a5_ecbd_44c1_a542_7a1cfb0fc369.slice/crio-3fe7b0923e90d88cfbc99fb4134f11a862c5c89447550fd58a54594cf9ec76ae WatchSource:0}: Error finding container 3fe7b0923e90d88cfbc99fb4134f11a862c5c89447550fd58a54594cf9ec76ae: Status 404 returned error can't find the container with id 3fe7b0923e90d88cfbc99fb4134f11a862c5c89447550fd58a54594cf9ec76ae
Sep 11 13:24:22.113405 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:22.113367    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-4h24k" event={"ID":"efcfccc2-da3a-41eb-96c7-31114c25ad5d","Type":"ContainerStarted","Data":"981402e430182dc273461a07609a9f5bc9076143fc3d24108a917b789f8278f1"}
Sep 11 13:24:22.113533 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:22.113435    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-4h24k"
Sep 11 13:24:22.115501 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:22.115441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56978c4d8d-m8lnd" event={"ID":"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369","Type":"ContainerStarted","Data":"3fe7b0923e90d88cfbc99fb4134f11a862c5c89447550fd58a54594cf9ec76ae"}
Sep 11 13:24:22.120133 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:22.120111    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:22.121531 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:22.121511    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-4h24k"
Sep 11 13:24:22.162220 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:22.162165    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-4h24k" podStartSLOduration=1.067495349 podStartE2EDuration="17.162145451s" totalImagesPullingTime="16.094650102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:24:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:22.136750355 +0000 UTC m=+181.355287917" watchObservedRunningTime="2026-09-11 13:24:22.162145451 +0000 UTC m=+181.380682956"
Sep 11 13:24:23.123537 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:23.123004    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56978c4d8d-m8lnd" event={"ID":"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369","Type":"ContainerStarted","Data":"ce9be3a7e6b499d670ecea28ee56453f6aa74bb42a3f26ac43b05757bc321403"}
Sep 11 13:24:23.145926 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:23.145843    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56978c4d8d-m8lnd" podStartSLOduration=2.145826786 podStartE2EDuration="2.145826786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:24:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:23.144509965 +0000 UTC m=+182.363047461" watchObservedRunningTime="2026-09-11 13:24:23.145826786 +0000 UTC m=+182.364364278"
Sep 11 13:24:29.152008 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:29.151972    2580 generic.go:356] "Generic (PLEG): container finished" podID="fa2dddcb-d99b-465d-be8f-d9db18b9555c" containerID="606a269b1bb2a597ef2eb6f9b00d492cb9d37440db927ca6fe48a2375076639d" exitCode=0
Sep 11 13:24:29.152484 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:29.152053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f" event={"ID":"fa2dddcb-d99b-465d-be8f-d9db18b9555c","Type":"ContainerDied","Data":"606a269b1bb2a597ef2eb6f9b00d492cb9d37440db927ca6fe48a2375076639d"}
Sep 11 13:24:29.152484 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:29.152439    2580 scope.go:117] "RemoveContainer" containerID="606a269b1bb2a597ef2eb6f9b00d492cb9d37440db927ca6fe48a2375076639d"
Sep 11 13:24:30.159006 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:30.158971    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-kjs8f" event={"ID":"fa2dddcb-d99b-465d-be8f-d9db18b9555c","Type":"ContainerStarted","Data":"19f24de6dd0c8a1f0662bd2d2115f92fcd1c681c1b14aacf46a6f418338b0e19"}
Sep 11 13:24:31.682979 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:31.682942    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:31.683352 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:31.682996    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:31.688206 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:31.688181    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:32.171583 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:32.171555    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:24:32.238688 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:32.238657    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f5447f885-msfnc"]
Sep 11 13:24:40.199663 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:40.199572    2580 generic.go:356] "Generic (PLEG): container finished" podID="24bb030f-2e95-4074-81ba-c28c6859db4c" containerID="8adef6b23ca51b95f021147a63aa63ce0a9c3de78ad2cf059487ee90872bb0c2" exitCode=0
Sep 11 13:24:40.200196 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:40.199648    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb" event={"ID":"24bb030f-2e95-4074-81ba-c28c6859db4c","Type":"ContainerDied","Data":"8adef6b23ca51b95f021147a63aa63ce0a9c3de78ad2cf059487ee90872bb0c2"}
Sep 11 13:24:40.200196 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:40.200082    2580 scope.go:117] "RemoveContainer" containerID="8adef6b23ca51b95f021147a63aa63ce0a9c3de78ad2cf059487ee90872bb0c2"
Sep 11 13:24:41.207015 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:41.206978    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rvdfb" event={"ID":"24bb030f-2e95-4074-81ba-c28c6859db4c","Type":"ContainerStarted","Data":"7b1d0f9aa4c3e9e0ca8e71d88a0e7aa7a07c1435ec4c2483e8ffe4a8e6092185"}
Sep 11 13:24:57.258110 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.258040    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f5447f885-msfnc" podUID="d5c871e9-28ba-47fc-9276-eba7713fd04b" containerName="console" containerID="cri-o://e61e2b913826110bddecd6c057a56bdd578070bbb19f36150a58ccee9d9d53ff" gracePeriod=15
Sep 11 13:24:57.531406 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.531330    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:57.660746 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.660713    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-oauth-config\") pod \"d5c871e9-28ba-47fc-9276-eba7713fd04b\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") "
Sep 11 13:24:57.660958 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.660769    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-config\") pod \"d5c871e9-28ba-47fc-9276-eba7713fd04b\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") "
Sep 11 13:24:57.660958 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.660819    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-oauth-serving-cert\") pod \"d5c871e9-28ba-47fc-9276-eba7713fd04b\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") "
Sep 11 13:24:57.660958 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.660864    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-service-ca\") pod \"d5c871e9-28ba-47fc-9276-eba7713fd04b\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") "
Sep 11 13:24:57.660958 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.660926    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-serving-cert\") pod \"d5c871e9-28ba-47fc-9276-eba7713fd04b\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") "
Sep 11 13:24:57.660958 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.660951    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhw86\" (UniqueName: \"kubernetes.io/projected/d5c871e9-28ba-47fc-9276-eba7713fd04b-kube-api-access-lhw86\") pod \"d5c871e9-28ba-47fc-9276-eba7713fd04b\" (UID: \"d5c871e9-28ba-47fc-9276-eba7713fd04b\") "
Sep 11 13:24:57.661322 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.661292    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-service-ca" (OuterVolumeSpecName: "service-ca") pod "d5c871e9-28ba-47fc-9276-eba7713fd04b" (UID: "d5c871e9-28ba-47fc-9276-eba7713fd04b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:57.661396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.661296    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-config" (OuterVolumeSpecName: "console-config") pod "d5c871e9-28ba-47fc-9276-eba7713fd04b" (UID: "d5c871e9-28ba-47fc-9276-eba7713fd04b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:57.661396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.661303    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d5c871e9-28ba-47fc-9276-eba7713fd04b" (UID: "d5c871e9-28ba-47fc-9276-eba7713fd04b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:57.663199 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.663167    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d5c871e9-28ba-47fc-9276-eba7713fd04b" (UID: "d5c871e9-28ba-47fc-9276-eba7713fd04b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:57.663303 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.663216    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d5c871e9-28ba-47fc-9276-eba7713fd04b" (UID: "d5c871e9-28ba-47fc-9276-eba7713fd04b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:57.663303 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.663216    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5c871e9-28ba-47fc-9276-eba7713fd04b-kube-api-access-lhw86" (OuterVolumeSpecName: "kube-api-access-lhw86") pod "d5c871e9-28ba-47fc-9276-eba7713fd04b" (UID: "d5c871e9-28ba-47fc-9276-eba7713fd04b"). InnerVolumeSpecName "kube-api-access-lhw86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:57.762364 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.762325    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-service-ca\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:24:57.762364 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.762357    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-serving-cert\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:24:57.762364 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.762367    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhw86\" (UniqueName: \"kubernetes.io/projected/d5c871e9-28ba-47fc-9276-eba7713fd04b-kube-api-access-lhw86\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:24:57.762364 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.762376    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-oauth-config\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:24:57.762619 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.762384    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-console-config\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:24:57.762619 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:57.762393    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5c871e9-28ba-47fc-9276-eba7713fd04b-oauth-serving-cert\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:24:58.282818 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:58.282776    2580 generic.go:356] "Generic (PLEG): container finished" podID="d5c871e9-28ba-47fc-9276-eba7713fd04b" containerID="e61e2b913826110bddecd6c057a56bdd578070bbb19f36150a58ccee9d9d53ff" exitCode=0
Sep 11 13:24:58.283239 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:58.282840    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f5447f885-msfnc"
Sep 11 13:24:58.283239 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:58.282857    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5447f885-msfnc" event={"ID":"d5c871e9-28ba-47fc-9276-eba7713fd04b","Type":"ContainerDied","Data":"e61e2b913826110bddecd6c057a56bdd578070bbb19f36150a58ccee9d9d53ff"}
Sep 11 13:24:58.283239 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:58.282952    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f5447f885-msfnc" event={"ID":"d5c871e9-28ba-47fc-9276-eba7713fd04b","Type":"ContainerDied","Data":"a06082dd6484144b656006a89f1f3597337f8eeb0bbdf49c566aec4a88245ce8"}
Sep 11 13:24:58.283239 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:58.282975    2580 scope.go:117] "RemoveContainer" containerID="e61e2b913826110bddecd6c057a56bdd578070bbb19f36150a58ccee9d9d53ff"
Sep 11 13:24:58.291488 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:58.291471    2580 scope.go:117] "RemoveContainer" containerID="e61e2b913826110bddecd6c057a56bdd578070bbb19f36150a58ccee9d9d53ff"
Sep 11 13:24:58.291798 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:24:58.291780    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e61e2b913826110bddecd6c057a56bdd578070bbb19f36150a58ccee9d9d53ff\": container with ID starting with e61e2b913826110bddecd6c057a56bdd578070bbb19f36150a58ccee9d9d53ff not found: ID does not exist" containerID="e61e2b913826110bddecd6c057a56bdd578070bbb19f36150a58ccee9d9d53ff"
Sep 11 13:24:58.291863 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:58.291810    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e61e2b913826110bddecd6c057a56bdd578070bbb19f36150a58ccee9d9d53ff"} err="failed to get container status \"e61e2b913826110bddecd6c057a56bdd578070bbb19f36150a58ccee9d9d53ff\": rpc error: code = NotFound desc = could not find container \"e61e2b913826110bddecd6c057a56bdd578070bbb19f36150a58ccee9d9d53ff\": container with ID starting with e61e2b913826110bddecd6c057a56bdd578070bbb19f36150a58ccee9d9d53ff not found: ID does not exist"
Sep 11 13:24:58.306279 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:58.306248    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f5447f885-msfnc"]
Sep 11 13:24:58.309739 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:58.309717    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f5447f885-msfnc"]
Sep 11 13:24:59.313856 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:24:59.313819    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5c871e9-28ba-47fc-9276-eba7713fd04b" path="/var/lib/kubelet/pods/d5c871e9-28ba-47fc-9276-eba7713fd04b/volumes"
Sep 11 13:25:36.129370 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.129335    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d748c9f7c-rvm8z"]
Sep 11 13:25:36.129745 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.129637    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5c871e9-28ba-47fc-9276-eba7713fd04b" containerName="console"
Sep 11 13:25:36.129745 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.129648    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5c871e9-28ba-47fc-9276-eba7713fd04b" containerName="console"
Sep 11 13:25:36.129745 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.129696    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5c871e9-28ba-47fc-9276-eba7713fd04b" containerName="console"
Sep 11 13:25:36.132017 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.132001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.166817 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.166781    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d748c9f7c-rvm8z"]
Sep 11 13:25:36.267155 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.267125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/48a9fd55-064f-4509-aad3-0e21020f8502-console-serving-cert\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.267155 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.267158    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-trusted-ca-bundle\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.267414 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.267180    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/48a9fd55-064f-4509-aad3-0e21020f8502-console-oauth-config\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.267414 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.267197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-service-ca\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.267414 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.267261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-oauth-serving-cert\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.267414 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.267332    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jbkr\" (UniqueName: \"kubernetes.io/projected/48a9fd55-064f-4509-aad3-0e21020f8502-kube-api-access-5jbkr\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.267414 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.267371    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-console-config\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.368693 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.368654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/48a9fd55-064f-4509-aad3-0e21020f8502-console-serving-cert\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.368693 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.368694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-trusted-ca-bundle\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.369000 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.368737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/48a9fd55-064f-4509-aad3-0e21020f8502-console-oauth-config\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.369000 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.368759    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-service-ca\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.369000 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.368787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-oauth-serving-cert\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.369000 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.368869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jbkr\" (UniqueName: \"kubernetes.io/projected/48a9fd55-064f-4509-aad3-0e21020f8502-kube-api-access-5jbkr\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.369000 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.368957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-console-config\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.369646 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.369600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-service-ca\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.369835 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.369813    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-oauth-serving-cert\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.369897 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.369876    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-console-config\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.369972 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.369898    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-trusted-ca-bundle\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.371304 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.371276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/48a9fd55-064f-4509-aad3-0e21020f8502-console-oauth-config\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.371393 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.371376    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/48a9fd55-064f-4509-aad3-0e21020f8502-console-serving-cert\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.378628 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.378606    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jbkr\" (UniqueName: \"kubernetes.io/projected/48a9fd55-064f-4509-aad3-0e21020f8502-kube-api-access-5jbkr\") pod \"console-7d748c9f7c-rvm8z\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") " pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.440685 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.440618    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:36.710453 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:36.710427    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d748c9f7c-rvm8z"]
Sep 11 13:25:36.712673 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:25:36.712646    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48a9fd55_064f_4509_aad3_0e21020f8502.slice/crio-3a9a3a56da64ad3e35962b268e985ed0d7159ea6ab2b3bd4f68c57e5560acf2b WatchSource:0}: Error finding container 3a9a3a56da64ad3e35962b268e985ed0d7159ea6ab2b3bd4f68c57e5560acf2b: Status 404 returned error can't find the container with id 3a9a3a56da64ad3e35962b268e985ed0d7159ea6ab2b3bd4f68c57e5560acf2b
Sep 11 13:25:37.429151 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:37.429115    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d748c9f7c-rvm8z" event={"ID":"48a9fd55-064f-4509-aad3-0e21020f8502","Type":"ContainerStarted","Data":"7420eb2ae3fe06e7244d97ab4bbe54d47f1626a94eee71e04a0a80dbe7c07d3b"}
Sep 11 13:25:37.429519 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:37.429163    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d748c9f7c-rvm8z" event={"ID":"48a9fd55-064f-4509-aad3-0e21020f8502","Type":"ContainerStarted","Data":"3a9a3a56da64ad3e35962b268e985ed0d7159ea6ab2b3bd4f68c57e5560acf2b"}
Sep 11 13:25:37.450722 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:37.450666    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d748c9f7c-rvm8z" podStartSLOduration=1.4506500469999999 podStartE2EDuration="1.450650047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:25:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:25:37.449523564 +0000 UTC m=+256.668061057" watchObservedRunningTime="2026-09-11 13:25:37.450650047 +0000 UTC m=+256.669187538"
Sep 11 13:25:46.441372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:46.441330    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:46.441746 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:46.441421    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:46.446016 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:46.445996    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:46.466191 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:46.466164    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:25:46.537146 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:25:46.537114    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56978c4d8d-m8lnd"]
Sep 11 13:26:11.556186 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.556122    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56978c4d8d-m8lnd" podUID="73bf35a5-ecbd-44c1-a542-7a1cfb0fc369" containerName="console" containerID="cri-o://ce9be3a7e6b499d670ecea28ee56453f6aa74bb42a3f26ac43b05757bc321403" gracePeriod=15
Sep 11 13:26:11.801514 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.801489    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:26:11.850424 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.850390    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-oauth-serving-cert\") pod \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") "
Sep 11 13:26:11.850587 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.850442    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-oauth-config\") pod \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") "
Sep 11 13:26:11.850587 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.850458    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gj2cz\" (UniqueName: \"kubernetes.io/projected/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-kube-api-access-gj2cz\") pod \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") "
Sep 11 13:26:11.850587 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.850490    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-serving-cert\") pod \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") "
Sep 11 13:26:11.850745 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.850678    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-trusted-ca-bundle\") pod \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") "
Sep 11 13:26:11.850745 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.850727    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-config\") pod \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") "
Sep 11 13:26:11.850855 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.850783    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-service-ca\") pod \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\" (UID: \"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369\") "
Sep 11 13:26:11.850855 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.850810    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "73bf35a5-ecbd-44c1-a542-7a1cfb0fc369" (UID: "73bf35a5-ecbd-44c1-a542-7a1cfb0fc369"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:26:11.851081 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.851055    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-oauth-serving-cert\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:26:11.851151 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.851091    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "73bf35a5-ecbd-44c1-a542-7a1cfb0fc369" (UID: "73bf35a5-ecbd-44c1-a542-7a1cfb0fc369"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:26:11.851210 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.851157    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-config" (OuterVolumeSpecName: "console-config") pod "73bf35a5-ecbd-44c1-a542-7a1cfb0fc369" (UID: "73bf35a5-ecbd-44c1-a542-7a1cfb0fc369"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:26:11.851268 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.851238    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-service-ca" (OuterVolumeSpecName: "service-ca") pod "73bf35a5-ecbd-44c1-a542-7a1cfb0fc369" (UID: "73bf35a5-ecbd-44c1-a542-7a1cfb0fc369"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:26:11.852691 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.852662    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-kube-api-access-gj2cz" (OuterVolumeSpecName: "kube-api-access-gj2cz") pod "73bf35a5-ecbd-44c1-a542-7a1cfb0fc369" (UID: "73bf35a5-ecbd-44c1-a542-7a1cfb0fc369"). InnerVolumeSpecName "kube-api-access-gj2cz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:26:11.852691 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.852681    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "73bf35a5-ecbd-44c1-a542-7a1cfb0fc369" (UID: "73bf35a5-ecbd-44c1-a542-7a1cfb0fc369"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:26:11.852817 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.852699    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "73bf35a5-ecbd-44c1-a542-7a1cfb0fc369" (UID: "73bf35a5-ecbd-44c1-a542-7a1cfb0fc369"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:26:11.951999 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.951960    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-oauth-config\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:26:11.951999 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.951991    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gj2cz\" (UniqueName: \"kubernetes.io/projected/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-kube-api-access-gj2cz\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:26:11.951999 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.952007    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-serving-cert\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:26:11.952242 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.952019    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-trusted-ca-bundle\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:26:11.952242 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.952031    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-console-config\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:26:11.952242 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:11.952043    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369-service-ca\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:26:12.560025 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:12.559991    2580 generic.go:356] "Generic (PLEG): container finished" podID="73bf35a5-ecbd-44c1-a542-7a1cfb0fc369" containerID="ce9be3a7e6b499d670ecea28ee56453f6aa74bb42a3f26ac43b05757bc321403" exitCode=0
Sep 11 13:26:12.560494 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:12.560047    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56978c4d8d-m8lnd"
Sep 11 13:26:12.560494 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:12.560074    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56978c4d8d-m8lnd" event={"ID":"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369","Type":"ContainerDied","Data":"ce9be3a7e6b499d670ecea28ee56453f6aa74bb42a3f26ac43b05757bc321403"}
Sep 11 13:26:12.560494 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:12.560122    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56978c4d8d-m8lnd" event={"ID":"73bf35a5-ecbd-44c1-a542-7a1cfb0fc369","Type":"ContainerDied","Data":"3fe7b0923e90d88cfbc99fb4134f11a862c5c89447550fd58a54594cf9ec76ae"}
Sep 11 13:26:12.560494 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:12.560148    2580 scope.go:117] "RemoveContainer" containerID="ce9be3a7e6b499d670ecea28ee56453f6aa74bb42a3f26ac43b05757bc321403"
Sep 11 13:26:12.569302 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:12.569284    2580 scope.go:117] "RemoveContainer" containerID="ce9be3a7e6b499d670ecea28ee56453f6aa74bb42a3f26ac43b05757bc321403"
Sep 11 13:26:12.569567 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:26:12.569548    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce9be3a7e6b499d670ecea28ee56453f6aa74bb42a3f26ac43b05757bc321403\": container with ID starting with ce9be3a7e6b499d670ecea28ee56453f6aa74bb42a3f26ac43b05757bc321403 not found: ID does not exist" containerID="ce9be3a7e6b499d670ecea28ee56453f6aa74bb42a3f26ac43b05757bc321403"
Sep 11 13:26:12.569630 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:12.569574    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce9be3a7e6b499d670ecea28ee56453f6aa74bb42a3f26ac43b05757bc321403"} err="failed to get container status \"ce9be3a7e6b499d670ecea28ee56453f6aa74bb42a3f26ac43b05757bc321403\": rpc error: code = NotFound desc = could not find container \"ce9be3a7e6b499d670ecea28ee56453f6aa74bb42a3f26ac43b05757bc321403\": container with ID starting with ce9be3a7e6b499d670ecea28ee56453f6aa74bb42a3f26ac43b05757bc321403 not found: ID does not exist"
Sep 11 13:26:12.583097 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:12.583079    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56978c4d8d-m8lnd"]
Sep 11 13:26:12.586130 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:12.586109    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56978c4d8d-m8lnd"]
Sep 11 13:26:13.314740 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:13.314706    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73bf35a5-ecbd-44c1-a542-7a1cfb0fc369" path="/var/lib/kubelet/pods/73bf35a5-ecbd-44c1-a542-7a1cfb0fc369/volumes"
Sep 11 13:26:21.223809 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:21.223781    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:26:21.226811 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:21.226787    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:26:21.241946 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:21.241919    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 13:26:23.280977 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.280940    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"]
Sep 11 13:26:23.282396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.281398    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73bf35a5-ecbd-44c1-a542-7a1cfb0fc369" containerName="console"
Sep 11 13:26:23.282396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.281417    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="73bf35a5-ecbd-44c1-a542-7a1cfb0fc369" containerName="console"
Sep 11 13:26:23.282396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.281474    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="73bf35a5-ecbd-44c1-a542-7a1cfb0fc369" containerName="console"
Sep 11 13:26:23.283343 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.283323    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"
Sep 11 13:26:23.289448 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.289424    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 13:26:23.289552 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.289424    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rh875\""
Sep 11 13:26:23.292256 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.292241    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 13:26:23.321397 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.321367    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"]
Sep 11 13:26:23.338137 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.338112    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h\" (UID: \"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"
Sep 11 13:26:23.338264 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.338240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h\" (UID: \"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"
Sep 11 13:26:23.338337 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.338309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xs89\" (UniqueName: \"kubernetes.io/projected/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-kube-api-access-7xs89\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h\" (UID: \"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"
Sep 11 13:26:23.439617 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.439582    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h\" (UID: \"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"
Sep 11 13:26:23.439784 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.439621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xs89\" (UniqueName: \"kubernetes.io/projected/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-kube-api-access-7xs89\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h\" (UID: \"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"
Sep 11 13:26:23.439784 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.439680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h\" (UID: \"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"
Sep 11 13:26:23.440067 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.440045    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h\" (UID: \"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"
Sep 11 13:26:23.440142 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.440057    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h\" (UID: \"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"
Sep 11 13:26:23.451689 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.451666    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xs89\" (UniqueName: \"kubernetes.io/projected/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-kube-api-access-7xs89\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h\" (UID: \"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"
Sep 11 13:26:23.592963 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.592941    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"
Sep 11 13:26:23.864242 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.864186    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"]
Sep 11 13:26:23.866460 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:26:23.866434    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45d1ece5_37ce_41d5_89ed_2f34ed2d6c4e.slice/crio-76b2786c422a1ef8674eb694ad9719997aaf439fddc6827f382fd967f0baa568 WatchSource:0}: Error finding container 76b2786c422a1ef8674eb694ad9719997aaf439fddc6827f382fd967f0baa568: Status 404 returned error can't find the container with id 76b2786c422a1ef8674eb694ad9719997aaf439fddc6827f382fd967f0baa568
Sep 11 13:26:23.868565 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:23.868545    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:26:24.609521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:24.609477    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h" event={"ID":"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e","Type":"ContainerStarted","Data":"76b2786c422a1ef8674eb694ad9719997aaf439fddc6827f382fd967f0baa568"}
Sep 11 13:26:30.637631 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:30.637592    2580 generic.go:356] "Generic (PLEG): container finished" podID="45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e" containerID="5c4cbb3054ddc4191cf467f5e4ea9f26bd3ec1e67675689f30dd33e153b878bd" exitCode=0
Sep 11 13:26:30.638085 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:30.637680    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h" event={"ID":"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e","Type":"ContainerDied","Data":"5c4cbb3054ddc4191cf467f5e4ea9f26bd3ec1e67675689f30dd33e153b878bd"}
Sep 11 13:26:38.670501 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:38.670468    2580 generic.go:356] "Generic (PLEG): container finished" podID="45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e" containerID="c00efedc16f3dab25a8bd17dcc8cfca739144946c1a8b07e80e7f4a525f9e285" exitCode=0
Sep 11 13:26:38.670858 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:38.670519    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h" event={"ID":"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e","Type":"ContainerDied","Data":"c00efedc16f3dab25a8bd17dcc8cfca739144946c1a8b07e80e7f4a525f9e285"}
Sep 11 13:26:45.702448 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:45.702415    2580 generic.go:356] "Generic (PLEG): container finished" podID="45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e" containerID="861699abded65fad12246de682e65fec21d94452118e084d7e39b597d74608ea" exitCode=0
Sep 11 13:26:45.702831 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:45.702478    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h" event={"ID":"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e","Type":"ContainerDied","Data":"861699abded65fad12246de682e65fec21d94452118e084d7e39b597d74608ea"}
Sep 11 13:26:46.830398 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:46.830374    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"
Sep 11 13:26:46.937111 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:46.937081    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-util\") pod \"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e\" (UID: \"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e\") "
Sep 11 13:26:46.937292 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:46.937143    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-bundle\") pod \"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e\" (UID: \"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e\") "
Sep 11 13:26:46.937292 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:46.937197    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xs89\" (UniqueName: \"kubernetes.io/projected/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-kube-api-access-7xs89\") pod \"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e\" (UID: \"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e\") "
Sep 11 13:26:46.937823 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:46.937796    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-bundle" (OuterVolumeSpecName: "bundle") pod "45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e" (UID: "45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:26:46.939394 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:46.939360    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-kube-api-access-7xs89" (OuterVolumeSpecName: "kube-api-access-7xs89") pod "45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e" (UID: "45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e"). InnerVolumeSpecName "kube-api-access-7xs89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:26:46.941492 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:46.941467    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-util" (OuterVolumeSpecName: "util") pod "45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e" (UID: "45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:26:47.038830 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:47.038745    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xs89\" (UniqueName: \"kubernetes.io/projected/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-kube-api-access-7xs89\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:26:47.038830 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:47.038784    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-util\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:26:47.038830 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:47.038799    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e-bundle\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:26:47.711563 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:47.711476    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h" event={"ID":"45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e","Type":"ContainerDied","Data":"76b2786c422a1ef8674eb694ad9719997aaf439fddc6827f382fd967f0baa568"}
Sep 11 13:26:47.711563 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:47.711512    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wd62h"
Sep 11 13:26:47.711563 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:47.711535    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76b2786c422a1ef8674eb694ad9719997aaf439fddc6827f382fd967f0baa568"
Sep 11 13:26:52.379209 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.379170    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9z6br"]
Sep 11 13:26:52.379569 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.379469    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e" containerName="util"
Sep 11 13:26:52.379569 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.379481    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e" containerName="util"
Sep 11 13:26:52.379569 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.379493    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e" containerName="pull"
Sep 11 13:26:52.379569 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.379501    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e" containerName="pull"
Sep 11 13:26:52.379569 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.379517    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e" containerName="extract"
Sep 11 13:26:52.379569 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.379523    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e" containerName="extract"
Sep 11 13:26:52.379754 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.379574    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="45d1ece5-37ce-41d5-89ed-2f34ed2d6c4e" containerName="extract"
Sep 11 13:26:52.381490 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.381473    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9z6br"
Sep 11 13:26:52.388391 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.388369    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 11 13:26:52.388524 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.388458    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-dxvg7\""
Sep 11 13:26:52.389589 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.389571    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:26:52.400167 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.400147    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9z6br"]
Sep 11 13:26:52.481566 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.481536    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1985143-d00e-4f08-8df0-c9b7eb1894ac-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9z6br\" (UID: \"d1985143-d00e-4f08-8df0-c9b7eb1894ac\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9z6br"
Sep 11 13:26:52.481732 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.481581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv7kn\" (UniqueName: \"kubernetes.io/projected/d1985143-d00e-4f08-8df0-c9b7eb1894ac-kube-api-access-xv7kn\") pod \"cert-manager-operator-controller-manager-55c444bf86-9z6br\" (UID: \"d1985143-d00e-4f08-8df0-c9b7eb1894ac\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9z6br"
Sep 11 13:26:52.581946 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.581889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv7kn\" (UniqueName: \"kubernetes.io/projected/d1985143-d00e-4f08-8df0-c9b7eb1894ac-kube-api-access-xv7kn\") pod \"cert-manager-operator-controller-manager-55c444bf86-9z6br\" (UID: \"d1985143-d00e-4f08-8df0-c9b7eb1894ac\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9z6br"
Sep 11 13:26:52.582091 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.582019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1985143-d00e-4f08-8df0-c9b7eb1894ac-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9z6br\" (UID: \"d1985143-d00e-4f08-8df0-c9b7eb1894ac\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9z6br"
Sep 11 13:26:52.582456 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.582435    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1985143-d00e-4f08-8df0-c9b7eb1894ac-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9z6br\" (UID: \"d1985143-d00e-4f08-8df0-c9b7eb1894ac\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9z6br"
Sep 11 13:26:52.595993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.595963    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv7kn\" (UniqueName: \"kubernetes.io/projected/d1985143-d00e-4f08-8df0-c9b7eb1894ac-kube-api-access-xv7kn\") pod \"cert-manager-operator-controller-manager-55c444bf86-9z6br\" (UID: \"d1985143-d00e-4f08-8df0-c9b7eb1894ac\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9z6br"
Sep 11 13:26:52.690077 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.689989    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9z6br"
Sep 11 13:26:52.968305 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:52.968282    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9z6br"]
Sep 11 13:26:52.971070 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:26:52.971045    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1985143_d00e_4f08_8df0_c9b7eb1894ac.slice/crio-8761959d48532197d94a8c0a11485ef8a59cbff399d2609998590e46e77aec40 WatchSource:0}: Error finding container 8761959d48532197d94a8c0a11485ef8a59cbff399d2609998590e46e77aec40: Status 404 returned error can't find the container with id 8761959d48532197d94a8c0a11485ef8a59cbff399d2609998590e46e77aec40
Sep 11 13:26:53.738972 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:53.738937    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9z6br" event={"ID":"d1985143-d00e-4f08-8df0-c9b7eb1894ac","Type":"ContainerStarted","Data":"8761959d48532197d94a8c0a11485ef8a59cbff399d2609998590e46e77aec40"}
Sep 11 13:26:54.745097 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:54.745012    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9z6br" event={"ID":"d1985143-d00e-4f08-8df0-c9b7eb1894ac","Type":"ContainerStarted","Data":"dc6636345b9aa903038eea486ea113a03be071b0454dc726215c067c1fcc2dcf"}
Sep 11 13:26:54.803724 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:54.803668    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9z6br" podStartSLOduration=1.400380095 podStartE2EDuration="2.803648053s" totalImagesPullingTime="1.403267958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:54.800804852 +0000 UTC m=+334.019342347" watchObservedRunningTime="2026-09-11 13:26:54.803648053 +0000 UTC m=+334.022185544"
Sep 11 13:26:55.466447 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.466409    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"]
Sep 11 13:26:55.470164 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.470140    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"
Sep 11 13:26:55.472731 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.472708    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rh875\""
Sep 11 13:26:55.472825 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.472743    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 13:26:55.473779 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.473763    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 13:26:55.483058 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.483036    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"]
Sep 11 13:26:55.607300 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.607266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zqzd\" (UniqueName: \"kubernetes.io/projected/95d7400c-c407-4529-9e84-1b62c758591e-kube-api-access-5zqzd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8\" (UID: \"95d7400c-c407-4529-9e84-1b62c758591e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"
Sep 11 13:26:55.607471 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.607322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95d7400c-c407-4529-9e84-1b62c758591e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8\" (UID: \"95d7400c-c407-4529-9e84-1b62c758591e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"
Sep 11 13:26:55.607471 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.607397    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95d7400c-c407-4529-9e84-1b62c758591e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8\" (UID: \"95d7400c-c407-4529-9e84-1b62c758591e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"
Sep 11 13:26:55.708406 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.708374    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95d7400c-c407-4529-9e84-1b62c758591e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8\" (UID: \"95d7400c-c407-4529-9e84-1b62c758591e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"
Sep 11 13:26:55.708565 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.708422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95d7400c-c407-4529-9e84-1b62c758591e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8\" (UID: \"95d7400c-c407-4529-9e84-1b62c758591e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"
Sep 11 13:26:55.708565 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.708476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zqzd\" (UniqueName: \"kubernetes.io/projected/95d7400c-c407-4529-9e84-1b62c758591e-kube-api-access-5zqzd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8\" (UID: \"95d7400c-c407-4529-9e84-1b62c758591e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"
Sep 11 13:26:55.708837 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.708814    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95d7400c-c407-4529-9e84-1b62c758591e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8\" (UID: \"95d7400c-c407-4529-9e84-1b62c758591e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"
Sep 11 13:26:55.708880 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.708826    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95d7400c-c407-4529-9e84-1b62c758591e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8\" (UID: \"95d7400c-c407-4529-9e84-1b62c758591e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"
Sep 11 13:26:55.719654 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.719600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zqzd\" (UniqueName: \"kubernetes.io/projected/95d7400c-c407-4529-9e84-1b62c758591e-kube-api-access-5zqzd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8\" (UID: \"95d7400c-c407-4529-9e84-1b62c758591e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"
Sep 11 13:26:55.779682 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:55.779637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"
Sep 11 13:26:56.056374 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.056182    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"]
Sep 11 13:26:56.058914 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:26:56.058872    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d7400c_c407_4529_9e84_1b62c758591e.slice/crio-19915b055a13ec2ec2f63f5f9130ed82f0dd03326c68309cba39185b5b0ec150 WatchSource:0}: Error finding container 19915b055a13ec2ec2f63f5f9130ed82f0dd03326c68309cba39185b5b0ec150: Status 404 returned error can't find the container with id 19915b055a13ec2ec2f63f5f9130ed82f0dd03326c68309cba39185b5b0ec150
Sep 11 13:26:56.331344 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.331309    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-x7lkr"]
Sep 11 13:26:56.334395 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.334375    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-x7lkr"
Sep 11 13:26:56.337465 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.337444    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 13:26:56.337582 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.337490    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-wrl8c\""
Sep 11 13:26:56.339290 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.339275    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 13:26:56.344264 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.344243    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-x7lkr"]
Sep 11 13:26:56.415218 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.415182    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxpnc\" (UniqueName: \"kubernetes.io/projected/9ba554b1-4c15-4353-bdd3-57edd351456b-kube-api-access-xxpnc\") pod \"cert-manager-webhook-6486ffd997-x7lkr\" (UID: \"9ba554b1-4c15-4353-bdd3-57edd351456b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7lkr"
Sep 11 13:26:56.415381 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.415227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ba554b1-4c15-4353-bdd3-57edd351456b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-x7lkr\" (UID: \"9ba554b1-4c15-4353-bdd3-57edd351456b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7lkr"
Sep 11 13:26:56.516298 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.516265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxpnc\" (UniqueName: \"kubernetes.io/projected/9ba554b1-4c15-4353-bdd3-57edd351456b-kube-api-access-xxpnc\") pod \"cert-manager-webhook-6486ffd997-x7lkr\" (UID: \"9ba554b1-4c15-4353-bdd3-57edd351456b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7lkr"
Sep 11 13:26:56.516298 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.516305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ba554b1-4c15-4353-bdd3-57edd351456b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-x7lkr\" (UID: \"9ba554b1-4c15-4353-bdd3-57edd351456b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7lkr"
Sep 11 13:26:56.530990 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.530961    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ba554b1-4c15-4353-bdd3-57edd351456b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-x7lkr\" (UID: \"9ba554b1-4c15-4353-bdd3-57edd351456b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7lkr"
Sep 11 13:26:56.531176 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.531157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxpnc\" (UniqueName: \"kubernetes.io/projected/9ba554b1-4c15-4353-bdd3-57edd351456b-kube-api-access-xxpnc\") pod \"cert-manager-webhook-6486ffd997-x7lkr\" (UID: \"9ba554b1-4c15-4353-bdd3-57edd351456b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x7lkr"
Sep 11 13:26:56.662448 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.662362    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-x7lkr"
Sep 11 13:26:56.755555 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.755527    2580 generic.go:356] "Generic (PLEG): container finished" podID="95d7400c-c407-4529-9e84-1b62c758591e" containerID="2ebae90298c341c9d87ac99531f3bb4fe7f00d4154aef40bb510d1652068feab" exitCode=0
Sep 11 13:26:56.755679 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.755582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8" event={"ID":"95d7400c-c407-4529-9e84-1b62c758591e","Type":"ContainerDied","Data":"2ebae90298c341c9d87ac99531f3bb4fe7f00d4154aef40bb510d1652068feab"}
Sep 11 13:26:56.755679 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.755630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8" event={"ID":"95d7400c-c407-4529-9e84-1b62c758591e","Type":"ContainerStarted","Data":"19915b055a13ec2ec2f63f5f9130ed82f0dd03326c68309cba39185b5b0ec150"}
Sep 11 13:26:56.933529 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:56.933502    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-x7lkr"]
Sep 11 13:26:56.935813 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:26:56.935785    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ba554b1_4c15_4353_bdd3_57edd351456b.slice/crio-dde4025aa96db8ffcf162c32169085d57e8da2c5b3b38b26146dd7d0bec38374 WatchSource:0}: Error finding container dde4025aa96db8ffcf162c32169085d57e8da2c5b3b38b26146dd7d0bec38374: Status 404 returned error can't find the container with id dde4025aa96db8ffcf162c32169085d57e8da2c5b3b38b26146dd7d0bec38374
Sep 11 13:26:57.763690 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:57.763646    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-x7lkr" event={"ID":"9ba554b1-4c15-4353-bdd3-57edd351456b","Type":"ContainerStarted","Data":"dde4025aa96db8ffcf162c32169085d57e8da2c5b3b38b26146dd7d0bec38374"}
Sep 11 13:26:58.770475 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:58.770384    2580 generic.go:356] "Generic (PLEG): container finished" podID="95d7400c-c407-4529-9e84-1b62c758591e" containerID="d9651bfc6b310c73bdb5ac698a10ed0809740fe906d7d2fc0cd5165e0dff77d9" exitCode=0
Sep 11 13:26:58.770939 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:58.770469    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8" event={"ID":"95d7400c-c407-4529-9e84-1b62c758591e","Type":"ContainerDied","Data":"d9651bfc6b310c73bdb5ac698a10ed0809740fe906d7d2fc0cd5165e0dff77d9"}
Sep 11 13:26:59.777766 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:59.777673    2580 generic.go:356] "Generic (PLEG): container finished" podID="95d7400c-c407-4529-9e84-1b62c758591e" containerID="248a03d155c0f51473d570219f7df033187397b2de17d348d1fc7d2ec7db8e8b" exitCode=0
Sep 11 13:26:59.778207 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:26:59.777760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8" event={"ID":"95d7400c-c407-4529-9e84-1b62c758591e","Type":"ContainerDied","Data":"248a03d155c0f51473d570219f7df033187397b2de17d348d1fc7d2ec7db8e8b"}
Sep 11 13:27:00.783779 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:00.783692    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-x7lkr" event={"ID":"9ba554b1-4c15-4353-bdd3-57edd351456b","Type":"ContainerStarted","Data":"f81e3ad1c021d8f2b96b3cd3a99df2742d366d95d25da28cf7766f5786921b81"}
Sep 11 13:27:00.784175 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:00.784013    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-x7lkr"
Sep 11 13:27:00.802500 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:00.802443    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-x7lkr" podStartSLOduration=1.392039956 podStartE2EDuration="4.802427416s" totalImagesPullingTime="3.41038746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:27:00.801242085 +0000 UTC m=+340.019779580" watchObservedRunningTime="2026-09-11 13:27:00.802427416 +0000 UTC m=+340.020964909"
Sep 11 13:27:00.910537 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:00.910510    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"
Sep 11 13:27:01.055760 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:01.055679    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95d7400c-c407-4529-9e84-1b62c758591e-bundle\") pod \"95d7400c-c407-4529-9e84-1b62c758591e\" (UID: \"95d7400c-c407-4529-9e84-1b62c758591e\") "
Sep 11 13:27:01.055760 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:01.055720    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zqzd\" (UniqueName: \"kubernetes.io/projected/95d7400c-c407-4529-9e84-1b62c758591e-kube-api-access-5zqzd\") pod \"95d7400c-c407-4529-9e84-1b62c758591e\" (UID: \"95d7400c-c407-4529-9e84-1b62c758591e\") "
Sep 11 13:27:01.055760 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:01.055743    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95d7400c-c407-4529-9e84-1b62c758591e-util\") pod \"95d7400c-c407-4529-9e84-1b62c758591e\" (UID: \"95d7400c-c407-4529-9e84-1b62c758591e\") "
Sep 11 13:27:01.056194 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:01.056169    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95d7400c-c407-4529-9e84-1b62c758591e-bundle" (OuterVolumeSpecName: "bundle") pod "95d7400c-c407-4529-9e84-1b62c758591e" (UID: "95d7400c-c407-4529-9e84-1b62c758591e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:27:01.057767 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:01.057743    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95d7400c-c407-4529-9e84-1b62c758591e-kube-api-access-5zqzd" (OuterVolumeSpecName: "kube-api-access-5zqzd") pod "95d7400c-c407-4529-9e84-1b62c758591e" (UID: "95d7400c-c407-4529-9e84-1b62c758591e"). InnerVolumeSpecName "kube-api-access-5zqzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:27:01.063711 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:01.063671    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95d7400c-c407-4529-9e84-1b62c758591e-util" (OuterVolumeSpecName: "util") pod "95d7400c-c407-4529-9e84-1b62c758591e" (UID: "95d7400c-c407-4529-9e84-1b62c758591e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:27:01.156624 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:01.156589    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95d7400c-c407-4529-9e84-1b62c758591e-bundle\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:27:01.156624 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:01.156620    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zqzd\" (UniqueName: \"kubernetes.io/projected/95d7400c-c407-4529-9e84-1b62c758591e-kube-api-access-5zqzd\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:27:01.156624 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:01.156630    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95d7400c-c407-4529-9e84-1b62c758591e-util\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:27:01.789756 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:01.789675    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8"
Sep 11 13:27:01.789756 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:01.789698    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftzxv8" event={"ID":"95d7400c-c407-4529-9e84-1b62c758591e","Type":"ContainerDied","Data":"19915b055a13ec2ec2f63f5f9130ed82f0dd03326c68309cba39185b5b0ec150"}
Sep 11 13:27:01.789756 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:01.789746    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19915b055a13ec2ec2f63f5f9130ed82f0dd03326c68309cba39185b5b0ec150"
Sep 11 13:27:06.792830 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:06.792799    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-x7lkr"
Sep 11 13:27:17.427384 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.427302    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"]
Sep 11 13:27:17.427807 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.427601    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95d7400c-c407-4529-9e84-1b62c758591e" containerName="util"
Sep 11 13:27:17.427807 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.427613    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="95d7400c-c407-4529-9e84-1b62c758591e" containerName="util"
Sep 11 13:27:17.427807 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.427621    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95d7400c-c407-4529-9e84-1b62c758591e" containerName="extract"
Sep 11 13:27:17.427807 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.427627    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="95d7400c-c407-4529-9e84-1b62c758591e" containerName="extract"
Sep 11 13:27:17.427807 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.427635    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95d7400c-c407-4529-9e84-1b62c758591e" containerName="pull"
Sep 11 13:27:17.427807 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.427640    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="95d7400c-c407-4529-9e84-1b62c758591e" containerName="pull"
Sep 11 13:27:17.427807 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.427693    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="95d7400c-c407-4529-9e84-1b62c758591e" containerName="extract"
Sep 11 13:27:17.449119 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.449079    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"]
Sep 11 13:27:17.449269 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.449201    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"
Sep 11 13:27:17.455009 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.454772    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rh875\""
Sep 11 13:27:17.455009 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.454773    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 13:27:17.455009 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.454873    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 13:27:17.481272 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.481241    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5fb73363-b348-408e-a804-a60bf451fc42-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb\" (UID: \"5fb73363-b348-408e-a804-a60bf451fc42\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"
Sep 11 13:27:17.481403 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.481309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs6pv\" (UniqueName: \"kubernetes.io/projected/5fb73363-b348-408e-a804-a60bf451fc42-kube-api-access-zs6pv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb\" (UID: \"5fb73363-b348-408e-a804-a60bf451fc42\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"
Sep 11 13:27:17.481403 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.481387    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5fb73363-b348-408e-a804-a60bf451fc42-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb\" (UID: \"5fb73363-b348-408e-a804-a60bf451fc42\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"
Sep 11 13:27:17.582247 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.582207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5fb73363-b348-408e-a804-a60bf451fc42-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb\" (UID: \"5fb73363-b348-408e-a804-a60bf451fc42\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"
Sep 11 13:27:17.582397 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.582299    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs6pv\" (UniqueName: \"kubernetes.io/projected/5fb73363-b348-408e-a804-a60bf451fc42-kube-api-access-zs6pv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb\" (UID: \"5fb73363-b348-408e-a804-a60bf451fc42\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"
Sep 11 13:27:17.582397 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.582364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5fb73363-b348-408e-a804-a60bf451fc42-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb\" (UID: \"5fb73363-b348-408e-a804-a60bf451fc42\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"
Sep 11 13:27:17.582600 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.582582    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5fb73363-b348-408e-a804-a60bf451fc42-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb\" (UID: \"5fb73363-b348-408e-a804-a60bf451fc42\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"
Sep 11 13:27:17.582734 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.582714    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5fb73363-b348-408e-a804-a60bf451fc42-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb\" (UID: \"5fb73363-b348-408e-a804-a60bf451fc42\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"
Sep 11 13:27:17.594334 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.594306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs6pv\" (UniqueName: \"kubernetes.io/projected/5fb73363-b348-408e-a804-a60bf451fc42-kube-api-access-zs6pv\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb\" (UID: \"5fb73363-b348-408e-a804-a60bf451fc42\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"
Sep 11 13:27:17.758303 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:17.758220    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"
Sep 11 13:27:18.052195 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:18.052167    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"]
Sep 11 13:27:18.053542 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:27:18.053516    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fb73363_b348_408e_a804_a60bf451fc42.slice/crio-188e7e7771a26a75b9fc2a15ddbc5a872b249ab1bc4331e798140314d040d7ef WatchSource:0}: Error finding container 188e7e7771a26a75b9fc2a15ddbc5a872b249ab1bc4331e798140314d040d7ef: Status 404 returned error can't find the container with id 188e7e7771a26a75b9fc2a15ddbc5a872b249ab1bc4331e798140314d040d7ef
Sep 11 13:27:18.865429 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:18.865341    2580 generic.go:356] "Generic (PLEG): container finished" podID="5fb73363-b348-408e-a804-a60bf451fc42" containerID="fad87f8a46cfa06237c85c9cc539528fb88b0b6b9720d5d6014f1d207cdea84b" exitCode=0
Sep 11 13:27:18.865429 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:18.865400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb" event={"ID":"5fb73363-b348-408e-a804-a60bf451fc42","Type":"ContainerDied","Data":"fad87f8a46cfa06237c85c9cc539528fb88b0b6b9720d5d6014f1d207cdea84b"}
Sep 11 13:27:18.865807 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:18.865464    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb" event={"ID":"5fb73363-b348-408e-a804-a60bf451fc42","Type":"ContainerStarted","Data":"188e7e7771a26a75b9fc2a15ddbc5a872b249ab1bc4331e798140314d040d7ef"}
Sep 11 13:27:19.871946 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:19.871894    2580 generic.go:356] "Generic (PLEG): container finished" podID="5fb73363-b348-408e-a804-a60bf451fc42" containerID="2b545625651bbe2e1b514c4b07f6f39ca50c74c7946c29c6fe5b351b01fe7a87" exitCode=0
Sep 11 13:27:19.872307 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:19.871973    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb" event={"ID":"5fb73363-b348-408e-a804-a60bf451fc42","Type":"ContainerDied","Data":"2b545625651bbe2e1b514c4b07f6f39ca50c74c7946c29c6fe5b351b01fe7a87"}
Sep 11 13:27:20.878072 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:20.878035    2580 generic.go:356] "Generic (PLEG): container finished" podID="5fb73363-b348-408e-a804-a60bf451fc42" containerID="ee7c22c1458801ee98f491ed22399688c0239c2ab283f9df0ea44f5cd31bb9bd" exitCode=0
Sep 11 13:27:20.878448 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:20.878121    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb" event={"ID":"5fb73363-b348-408e-a804-a60bf451fc42","Type":"ContainerDied","Data":"ee7c22c1458801ee98f491ed22399688c0239c2ab283f9df0ea44f5cd31bb9bd"}
Sep 11 13:27:22.001135 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:22.001113    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"
Sep 11 13:27:22.016222 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:22.016201    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5fb73363-b348-408e-a804-a60bf451fc42-util\") pod \"5fb73363-b348-408e-a804-a60bf451fc42\" (UID: \"5fb73363-b348-408e-a804-a60bf451fc42\") "
Sep 11 13:27:22.016333 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:22.016237    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5fb73363-b348-408e-a804-a60bf451fc42-bundle\") pod \"5fb73363-b348-408e-a804-a60bf451fc42\" (UID: \"5fb73363-b348-408e-a804-a60bf451fc42\") "
Sep 11 13:27:22.016333 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:22.016285    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs6pv\" (UniqueName: \"kubernetes.io/projected/5fb73363-b348-408e-a804-a60bf451fc42-kube-api-access-zs6pv\") pod \"5fb73363-b348-408e-a804-a60bf451fc42\" (UID: \"5fb73363-b348-408e-a804-a60bf451fc42\") "
Sep 11 13:27:22.016898 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:22.016875    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fb73363-b348-408e-a804-a60bf451fc42-bundle" (OuterVolumeSpecName: "bundle") pod "5fb73363-b348-408e-a804-a60bf451fc42" (UID: "5fb73363-b348-408e-a804-a60bf451fc42"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:27:22.018386 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:22.018364    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fb73363-b348-408e-a804-a60bf451fc42-kube-api-access-zs6pv" (OuterVolumeSpecName: "kube-api-access-zs6pv") pod "5fb73363-b348-408e-a804-a60bf451fc42" (UID: "5fb73363-b348-408e-a804-a60bf451fc42"). InnerVolumeSpecName "kube-api-access-zs6pv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:27:22.021679 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:22.021658    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fb73363-b348-408e-a804-a60bf451fc42-util" (OuterVolumeSpecName: "util") pod "5fb73363-b348-408e-a804-a60bf451fc42" (UID: "5fb73363-b348-408e-a804-a60bf451fc42"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:27:22.117347 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:22.117311    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5fb73363-b348-408e-a804-a60bf451fc42-util\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:27:22.117347 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:22.117338    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5fb73363-b348-408e-a804-a60bf451fc42-bundle\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:27:22.117347 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:22.117353    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs6pv\" (UniqueName: \"kubernetes.io/projected/5fb73363-b348-408e-a804-a60bf451fc42-kube-api-access-zs6pv\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:27:22.889453 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:22.889425    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb"
Sep 11 13:27:22.889655 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:22.889425    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcnfjb" event={"ID":"5fb73363-b348-408e-a804-a60bf451fc42","Type":"ContainerDied","Data":"188e7e7771a26a75b9fc2a15ddbc5a872b249ab1bc4331e798140314d040d7ef"}
Sep 11 13:27:22.889655 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:22.889548    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="188e7e7771a26a75b9fc2a15ddbc5a872b249ab1bc4331e798140314d040d7ef"
Sep 11 13:27:28.369262 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.369220    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"]
Sep 11 13:27:28.369711 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.369652    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb73363-b348-408e-a804-a60bf451fc42" containerName="extract"
Sep 11 13:27:28.369711 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.369672    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb73363-b348-408e-a804-a60bf451fc42" containerName="extract"
Sep 11 13:27:28.369711 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.369691    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb73363-b348-408e-a804-a60bf451fc42" containerName="pull"
Sep 11 13:27:28.369711 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.369698    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb73363-b348-408e-a804-a60bf451fc42" containerName="pull"
Sep 11 13:27:28.369711 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.369712    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb73363-b348-408e-a804-a60bf451fc42" containerName="util"
Sep 11 13:27:28.370013 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.369720    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb73363-b348-408e-a804-a60bf451fc42" containerName="util"
Sep 11 13:27:28.370013 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.369808    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fb73363-b348-408e-a804-a60bf451fc42" containerName="extract"
Sep 11 13:27:28.374063 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.374040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"
Sep 11 13:27:28.376750 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.376731    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 13:27:28.377991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.377974    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rh875\""
Sep 11 13:27:28.377991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.377988    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 13:27:28.386103 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.386078    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"]
Sep 11 13:27:28.468705 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.468661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp\" (UID: \"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"
Sep 11 13:27:28.468874 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.468765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqbgn\" (UniqueName: \"kubernetes.io/projected/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-kube-api-access-gqbgn\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp\" (UID: \"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"
Sep 11 13:27:28.468874 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.468833    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp\" (UID: \"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"
Sep 11 13:27:28.569545 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.569512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqbgn\" (UniqueName: \"kubernetes.io/projected/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-kube-api-access-gqbgn\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp\" (UID: \"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"
Sep 11 13:27:28.569748 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.569583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp\" (UID: \"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"
Sep 11 13:27:28.569748 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.569622    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp\" (UID: \"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"
Sep 11 13:27:28.569985 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.569963    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp\" (UID: \"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"
Sep 11 13:27:28.570051 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.570006    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp\" (UID: \"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"
Sep 11 13:27:28.578653 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.578633    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqbgn\" (UniqueName: \"kubernetes.io/projected/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-kube-api-access-gqbgn\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp\" (UID: \"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"
Sep 11 13:27:28.683057 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.682957    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"
Sep 11 13:27:28.966890 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:28.966812    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"]
Sep 11 13:27:28.970499 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:27:28.970464    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a9c6cea_fd80_4cce_97a3_cc285f3bbcc0.slice/crio-99bba47b4684b29401e74b5cc1517c9d85cc56d1caec5ea65849f09be16475bf WatchSource:0}: Error finding container 99bba47b4684b29401e74b5cc1517c9d85cc56d1caec5ea65849f09be16475bf: Status 404 returned error can't find the container with id 99bba47b4684b29401e74b5cc1517c9d85cc56d1caec5ea65849f09be16475bf
Sep 11 13:27:29.923380 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:29.923291    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0" containerID="c542a20b3c959ed3e38286ed975fbaf14ce74275ac4dd60088c7cd1491a3d182" exitCode=0
Sep 11 13:27:29.923380 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:29.923354    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp" event={"ID":"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0","Type":"ContainerDied","Data":"c542a20b3c959ed3e38286ed975fbaf14ce74275ac4dd60088c7cd1491a3d182"}
Sep 11 13:27:29.923814 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:29.923395    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp" event={"ID":"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0","Type":"ContainerStarted","Data":"99bba47b4684b29401e74b5cc1517c9d85cc56d1caec5ea65849f09be16475bf"}
Sep 11 13:27:30.552557 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.552524    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"]
Sep 11 13:27:30.573583 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.573559    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.576885 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.576863    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 13:27:30.577019 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.576869    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 13:27:30.577019 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.576869    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 13:27:30.577019 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.576945    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2ksm4\""
Sep 11 13:27:30.577297 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.577276    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 13:27:30.582348 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.582322    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"]
Sep 11 13:27:30.688353 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.688268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5jv2\" (UniqueName: \"kubernetes.io/projected/4b978d74-2efb-4262-b0a0-ad66cc441e03-kube-api-access-f5jv2\") pod \"opendatahub-operator-controller-manager-9576ccb85-fkn2f\" (UID: \"4b978d74-2efb-4262-b0a0-ad66cc441e03\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.688353 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.688324    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b978d74-2efb-4262-b0a0-ad66cc441e03-tmp\") pod \"opendatahub-operator-controller-manager-9576ccb85-fkn2f\" (UID: \"4b978d74-2efb-4262-b0a0-ad66cc441e03\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.688528 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.688381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4b978d74-2efb-4262-b0a0-ad66cc441e03-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9576ccb85-fkn2f\" (UID: \"4b978d74-2efb-4262-b0a0-ad66cc441e03\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.688528 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.688434    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4b978d74-2efb-4262-b0a0-ad66cc441e03-webhook-cert\") pod \"opendatahub-operator-controller-manager-9576ccb85-fkn2f\" (UID: \"4b978d74-2efb-4262-b0a0-ad66cc441e03\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.688528 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.688452    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4b978d74-2efb-4262-b0a0-ad66cc441e03-manifests\") pod \"opendatahub-operator-controller-manager-9576ccb85-fkn2f\" (UID: \"4b978d74-2efb-4262-b0a0-ad66cc441e03\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.789219 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.789182    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4b978d74-2efb-4262-b0a0-ad66cc441e03-webhook-cert\") pod \"opendatahub-operator-controller-manager-9576ccb85-fkn2f\" (UID: \"4b978d74-2efb-4262-b0a0-ad66cc441e03\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.789219 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.789219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4b978d74-2efb-4262-b0a0-ad66cc441e03-manifests\") pod \"opendatahub-operator-controller-manager-9576ccb85-fkn2f\" (UID: \"4b978d74-2efb-4262-b0a0-ad66cc441e03\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.789418 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.789262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5jv2\" (UniqueName: \"kubernetes.io/projected/4b978d74-2efb-4262-b0a0-ad66cc441e03-kube-api-access-f5jv2\") pod \"opendatahub-operator-controller-manager-9576ccb85-fkn2f\" (UID: \"4b978d74-2efb-4262-b0a0-ad66cc441e03\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.789418 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.789294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b978d74-2efb-4262-b0a0-ad66cc441e03-tmp\") pod \"opendatahub-operator-controller-manager-9576ccb85-fkn2f\" (UID: \"4b978d74-2efb-4262-b0a0-ad66cc441e03\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.789418 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.789313    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4b978d74-2efb-4262-b0a0-ad66cc441e03-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9576ccb85-fkn2f\" (UID: \"4b978d74-2efb-4262-b0a0-ad66cc441e03\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.789793 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.789765    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4b978d74-2efb-4262-b0a0-ad66cc441e03-manifests\") pod \"opendatahub-operator-controller-manager-9576ccb85-fkn2f\" (UID: \"4b978d74-2efb-4262-b0a0-ad66cc441e03\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.791675 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.791651    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b978d74-2efb-4262-b0a0-ad66cc441e03-tmp\") pod \"opendatahub-operator-controller-manager-9576ccb85-fkn2f\" (UID: \"4b978d74-2efb-4262-b0a0-ad66cc441e03\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.791779 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.791734    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4b978d74-2efb-4262-b0a0-ad66cc441e03-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9576ccb85-fkn2f\" (UID: \"4b978d74-2efb-4262-b0a0-ad66cc441e03\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.791822 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.791797    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4b978d74-2efb-4262-b0a0-ad66cc441e03-webhook-cert\") pod \"opendatahub-operator-controller-manager-9576ccb85-fkn2f\" (UID: \"4b978d74-2efb-4262-b0a0-ad66cc441e03\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.803632 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.803602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5jv2\" (UniqueName: \"kubernetes.io/projected/4b978d74-2efb-4262-b0a0-ad66cc441e03-kube-api-access-f5jv2\") pod \"opendatahub-operator-controller-manager-9576ccb85-fkn2f\" (UID: \"4b978d74-2efb-4262-b0a0-ad66cc441e03\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:30.931004 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.930970    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0" containerID="e0219d8bf71435d26a36a1bf085887bd73bff387fe85f55a51d3c30659a81c92" exitCode=0
Sep 11 13:27:30.931383 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.931063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp" event={"ID":"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0","Type":"ContainerDied","Data":"e0219d8bf71435d26a36a1bf085887bd73bff387fe85f55a51d3c30659a81c92"}
Sep 11 13:27:30.943317 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:30.943262    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:31.257262 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:31.257183    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"]
Sep 11 13:27:31.260454 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:27:31.260424    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b978d74_2efb_4262_b0a0_ad66cc441e03.slice/crio-0a829bf206221f94ad0b2dd1f9018f7a1a01f3c72a9bed74faec37bd1f0ec84d WatchSource:0}: Error finding container 0a829bf206221f94ad0b2dd1f9018f7a1a01f3c72a9bed74faec37bd1f0ec84d: Status 404 returned error can't find the container with id 0a829bf206221f94ad0b2dd1f9018f7a1a01f3c72a9bed74faec37bd1f0ec84d
Sep 11 13:27:31.940319 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:31.940281    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0" containerID="1ba4df160254075e8ae0194b853df93ce6e8888ce1678a529c80aba4636f407b" exitCode=0
Sep 11 13:27:31.940781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:31.940342    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp" event={"ID":"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0","Type":"ContainerDied","Data":"1ba4df160254075e8ae0194b853df93ce6e8888ce1678a529c80aba4636f407b"}
Sep 11 13:27:31.942474 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:31.942440    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f" event={"ID":"4b978d74-2efb-4262-b0a0-ad66cc441e03","Type":"ContainerStarted","Data":"0a829bf206221f94ad0b2dd1f9018f7a1a01f3c72a9bed74faec37bd1f0ec84d"}
Sep 11 13:27:33.961960 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:33.961929    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp" event={"ID":"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0","Type":"ContainerDied","Data":"99bba47b4684b29401e74b5cc1517c9d85cc56d1caec5ea65849f09be16475bf"}
Sep 11 13:27:33.962262 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:33.962010    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99bba47b4684b29401e74b5cc1517c9d85cc56d1caec5ea65849f09be16475bf"
Sep 11 13:27:33.967959 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:33.967939    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"
Sep 11 13:27:34.017356 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:34.017333    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqbgn\" (UniqueName: \"kubernetes.io/projected/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-kube-api-access-gqbgn\") pod \"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0\" (UID: \"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0\") "
Sep 11 13:27:34.017458 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:34.017410    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-util\") pod \"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0\" (UID: \"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0\") "
Sep 11 13:27:34.017518 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:34.017458    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-bundle\") pod \"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0\" (UID: \"1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0\") "
Sep 11 13:27:34.018142 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:34.018118    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-bundle" (OuterVolumeSpecName: "bundle") pod "1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0" (UID: "1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:27:34.019379 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:34.019352    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-kube-api-access-gqbgn" (OuterVolumeSpecName: "kube-api-access-gqbgn") pod "1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0" (UID: "1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0"). InnerVolumeSpecName "kube-api-access-gqbgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:27:34.022845 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:34.022818    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-util" (OuterVolumeSpecName: "util") pod "1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0" (UID: "1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:27:34.118076 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:34.118043    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-bundle\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:27:34.118076 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:34.118072    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqbgn\" (UniqueName: \"kubernetes.io/projected/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-kube-api-access-gqbgn\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:27:34.118076 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:34.118082    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0-util\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:27:34.968060 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:34.968020    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f" event={"ID":"4b978d74-2efb-4262-b0a0-ad66cc441e03","Type":"ContainerStarted","Data":"73d50af9624a634fd3ad4875c593f6c641f5fd3dac61423296ce50da20f5cf1d"}
Sep 11 13:27:34.968423 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:34.968108    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mx8wp"
Sep 11 13:27:35.974213 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:35.974179    2580 generic.go:356] "Generic (PLEG): container finished" podID="4b978d74-2efb-4262-b0a0-ad66cc441e03" containerID="73d50af9624a634fd3ad4875c593f6c641f5fd3dac61423296ce50da20f5cf1d" exitCode=0
Sep 11 13:27:35.974582 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:35.974269    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f" event={"ID":"4b978d74-2efb-4262-b0a0-ad66cc441e03","Type":"ContainerDied","Data":"73d50af9624a634fd3ad4875c593f6c641f5fd3dac61423296ce50da20f5cf1d"}
Sep 11 13:27:36.981802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:36.981759    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f" event={"ID":"4b978d74-2efb-4262-b0a0-ad66cc441e03","Type":"ContainerStarted","Data":"d0bd650afbd2f4e12b220eee386b8b877fafaca7dd04a92d0eb91f8daacd291a"}
Sep 11 13:27:36.982222 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:36.981890    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:37.009842 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:37.009797    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f" podStartSLOduration=2.8911193280000003 podStartE2EDuration="7.009781715s" totalImagesPullingTime="3.095434873s" totalInitContainerRuntime="1.023227514s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:27:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:27:37.009677794 +0000 UTC m=+376.228215286" watchObservedRunningTime="2026-09-11 13:27:37.009781715 +0000 UTC m=+376.228319206"
Sep 11 13:27:47.989453 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:47.989424    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-fkn2f"
Sep 11 13:27:49.280599 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.280566    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"]
Sep 11 13:27:49.281110 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.280864    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0" containerName="extract"
Sep 11 13:27:49.281110 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.280875    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0" containerName="extract"
Sep 11 13:27:49.281110 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.280887    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0" containerName="util"
Sep 11 13:27:49.281110 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.280892    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0" containerName="util"
Sep 11 13:27:49.281110 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.280917    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0" containerName="pull"
Sep 11 13:27:49.281110 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.280924    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0" containerName="pull"
Sep 11 13:27:49.281110 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.280998    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a9c6cea-fd80-4cce-97a3-cc285f3bbcc0" containerName="extract"
Sep 11 13:27:49.283968 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.283951    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:27:49.287494 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.287472    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 13:27:49.288444 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.288415    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 13:27:49.288615 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.288480    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xjglx\""
Sep 11 13:27:49.289009 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.288979    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 13:27:49.289130 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.289020    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:27:49.291864 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.291847    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 13:27:49.301029 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.301007    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"]
Sep 11 13:27:49.348177 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.348142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4-metrics-cert\") pod \"lws-controller-manager-84bdbdc8f4-kfqct\" (UID: \"9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:27:49.348177 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.348178    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2z55\" (UniqueName: \"kubernetes.io/projected/9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4-kube-api-access-l2z55\") pod \"lws-controller-manager-84bdbdc8f4-kfqct\" (UID: \"9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:27:49.348421 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.348201    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4-cert\") pod \"lws-controller-manager-84bdbdc8f4-kfqct\" (UID: \"9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:27:49.348421 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.348322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4-manager-config\") pod \"lws-controller-manager-84bdbdc8f4-kfqct\" (UID: \"9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:27:49.449712 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.449664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4-metrics-cert\") pod \"lws-controller-manager-84bdbdc8f4-kfqct\" (UID: \"9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:27:49.449712 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.449713    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2z55\" (UniqueName: \"kubernetes.io/projected/9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4-kube-api-access-l2z55\") pod \"lws-controller-manager-84bdbdc8f4-kfqct\" (UID: \"9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:27:49.449993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.449735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4-cert\") pod \"lws-controller-manager-84bdbdc8f4-kfqct\" (UID: \"9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:27:49.449993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.449780    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4-manager-config\") pod \"lws-controller-manager-84bdbdc8f4-kfqct\" (UID: \"9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:27:49.450497 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.450476    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4-manager-config\") pod \"lws-controller-manager-84bdbdc8f4-kfqct\" (UID: \"9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:27:49.452166 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.452142    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4-cert\") pod \"lws-controller-manager-84bdbdc8f4-kfqct\" (UID: \"9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:27:49.452335 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.452301    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4-metrics-cert\") pod \"lws-controller-manager-84bdbdc8f4-kfqct\" (UID: \"9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:27:49.461517 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.461492    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2z55\" (UniqueName: \"kubernetes.io/projected/9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4-kube-api-access-l2z55\") pod \"lws-controller-manager-84bdbdc8f4-kfqct\" (UID: \"9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:27:49.593046 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.593010    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:27:49.910756 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:49.910732    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"]
Sep 11 13:27:49.912898 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:27:49.912871    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f3cf67b_7fa7_4f98_97cf_6ba1f951cce4.slice/crio-69676fba3160b9e643e60c60d4c85d6dcfe4c20e340beb6361f7e6a59a0065eb WatchSource:0}: Error finding container 69676fba3160b9e643e60c60d4c85d6dcfe4c20e340beb6361f7e6a59a0065eb: Status 404 returned error can't find the container with id 69676fba3160b9e643e60c60d4c85d6dcfe4c20e340beb6361f7e6a59a0065eb
Sep 11 13:27:50.046961 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.046926    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct" event={"ID":"9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4","Type":"ContainerStarted","Data":"69676fba3160b9e643e60c60d4c85d6dcfe4c20e340beb6361f7e6a59a0065eb"}
Sep 11 13:27:50.351126 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.351089    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f"]
Sep 11 13:27:50.357034 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.357007    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f"
Sep 11 13:27:50.360061 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.360032    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 13:27:50.360216 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.360203    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 13:27:50.360362 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.360338    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mm6pm\""
Sep 11 13:27:50.361290 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.361272    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 13:27:50.361406 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.361301    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 13:27:50.370580 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.370557    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f"]
Sep 11 13:27:50.458653 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.458615    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e7c7651-a31f-413e-99d5-3e61b09e46d5-tls-certs\") pod \"kube-auth-proxy-7c9bdb7954-t6r2f\" (UID: \"7e7c7651-a31f-413e-99d5-3e61b09e46d5\") " pod="openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f"
Sep 11 13:27:50.458824 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.458705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e7c7651-a31f-413e-99d5-3e61b09e46d5-tmp\") pod \"kube-auth-proxy-7c9bdb7954-t6r2f\" (UID: \"7e7c7651-a31f-413e-99d5-3e61b09e46d5\") " pod="openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f"
Sep 11 13:27:50.458824 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.458740    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbx5c\" (UniqueName: \"kubernetes.io/projected/7e7c7651-a31f-413e-99d5-3e61b09e46d5-kube-api-access-dbx5c\") pod \"kube-auth-proxy-7c9bdb7954-t6r2f\" (UID: \"7e7c7651-a31f-413e-99d5-3e61b09e46d5\") " pod="openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f"
Sep 11 13:27:50.559976 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.559931    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e7c7651-a31f-413e-99d5-3e61b09e46d5-tmp\") pod \"kube-auth-proxy-7c9bdb7954-t6r2f\" (UID: \"7e7c7651-a31f-413e-99d5-3e61b09e46d5\") " pod="openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f"
Sep 11 13:27:50.560188 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.560033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbx5c\" (UniqueName: \"kubernetes.io/projected/7e7c7651-a31f-413e-99d5-3e61b09e46d5-kube-api-access-dbx5c\") pod \"kube-auth-proxy-7c9bdb7954-t6r2f\" (UID: \"7e7c7651-a31f-413e-99d5-3e61b09e46d5\") " pod="openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f"
Sep 11 13:27:50.560188 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.560126    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e7c7651-a31f-413e-99d5-3e61b09e46d5-tls-certs\") pod \"kube-auth-proxy-7c9bdb7954-t6r2f\" (UID: \"7e7c7651-a31f-413e-99d5-3e61b09e46d5\") " pod="openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f"
Sep 11 13:27:50.562340 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.562319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e7c7651-a31f-413e-99d5-3e61b09e46d5-tmp\") pod \"kube-auth-proxy-7c9bdb7954-t6r2f\" (UID: \"7e7c7651-a31f-413e-99d5-3e61b09e46d5\") " pod="openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f"
Sep 11 13:27:50.562680 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.562658    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e7c7651-a31f-413e-99d5-3e61b09e46d5-tls-certs\") pod \"kube-auth-proxy-7c9bdb7954-t6r2f\" (UID: \"7e7c7651-a31f-413e-99d5-3e61b09e46d5\") " pod="openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f"
Sep 11 13:27:50.591377 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.591339    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbx5c\" (UniqueName: \"kubernetes.io/projected/7e7c7651-a31f-413e-99d5-3e61b09e46d5-kube-api-access-dbx5c\") pod \"kube-auth-proxy-7c9bdb7954-t6r2f\" (UID: \"7e7c7651-a31f-413e-99d5-3e61b09e46d5\") " pod="openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f"
Sep 11 13:27:50.667398 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.667316    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f"
Sep 11 13:27:50.957756 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:27:50.957725    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e7c7651_a31f_413e_99d5_3e61b09e46d5.slice/crio-8b3da21f8feaf5071f762315f30ef86c0efd4a3e8a9ee8e0a9bca6583f528fd5 WatchSource:0}: Error finding container 8b3da21f8feaf5071f762315f30ef86c0efd4a3e8a9ee8e0a9bca6583f528fd5: Status 404 returned error can't find the container with id 8b3da21f8feaf5071f762315f30ef86c0efd4a3e8a9ee8e0a9bca6583f528fd5
Sep 11 13:27:50.958851 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:50.958822    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f"]
Sep 11 13:27:51.054668 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:51.054630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f" event={"ID":"7e7c7651-a31f-413e-99d5-3e61b09e46d5","Type":"ContainerStarted","Data":"8b3da21f8feaf5071f762315f30ef86c0efd4a3e8a9ee8e0a9bca6583f528fd5"}
Sep 11 13:27:52.064770 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:52.064719    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct" event={"ID":"9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4","Type":"ContainerStarted","Data":"ae22cb01922aff2108fd19e46dfd19b86202f3074a113e94fe39d539db45bdae"}
Sep 11 13:27:52.065411 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:52.065380    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:27:52.087607 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:52.087555    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct" podStartSLOduration=1.075695827 podStartE2EDuration="3.087534791s" totalImagesPullingTime="2.011838964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:27:52.086785822 +0000 UTC m=+391.305323321" watchObservedRunningTime="2026-09-11 13:27:52.087534791 +0000 UTC m=+391.306072285"
Sep 11 13:27:54.076144 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:54.076107    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f" event={"ID":"7e7c7651-a31f-413e-99d5-3e61b09e46d5","Type":"ContainerStarted","Data":"2a9657a483f046ca802ff8d1700cfe0920c566ff67216fa3d1cda6b88316d713"}
Sep 11 13:27:54.095127 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:27:54.095076    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c9bdb7954-t6r2f" podStartSLOduration=1.5509407149999999 podStartE2EDuration="4.095061628s" totalImagesPullingTime="2.544120913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:27:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:27:54.094067836 +0000 UTC m=+393.312605329" watchObservedRunningTime="2026-09-11 13:27:54.095061628 +0000 UTC m=+393.313599120"
Sep 11 13:28:03.072533 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:28:03.072493    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-kfqct"
Sep 11 13:29:07.958597 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:07.958564    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"]
Sep 11 13:29:07.961270 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:07.961253    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"
Sep 11 13:29:07.965989 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:07.965968    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 13:29:07.966112 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:07.965972    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 13:29:07.966112 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:07.966039    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rh875\""
Sep 11 13:29:07.973894 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:07.973872    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"]
Sep 11 13:29:08.044740 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.044708    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"]
Sep 11 13:29:08.047045 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.047029    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"
Sep 11 13:29:08.063243 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.063215    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"]
Sep 11 13:29:08.089632 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.089601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ca978df-3ba4-4a91-b512-81e1fdc7a397-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp\" (UID: \"8ca978df-3ba4-4a91-b512-81e1fdc7a397\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"
Sep 11 13:29:08.089788 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.089642    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lpj8\" (UniqueName: \"kubernetes.io/projected/8ca978df-3ba4-4a91-b512-81e1fdc7a397-kube-api-access-8lpj8\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp\" (UID: \"8ca978df-3ba4-4a91-b512-81e1fdc7a397\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"
Sep 11 13:29:08.089788 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.089664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ca978df-3ba4-4a91-b512-81e1fdc7a397-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp\" (UID: \"8ca978df-3ba4-4a91-b512-81e1fdc7a397\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"
Sep 11 13:29:08.130966 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.130935    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"]
Sep 11 13:29:08.133366 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.133342    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"
Sep 11 13:29:08.143603 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.143581    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"]
Sep 11 13:29:08.190718 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.190684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ca978df-3ba4-4a91-b512-81e1fdc7a397-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp\" (UID: \"8ca978df-3ba4-4a91-b512-81e1fdc7a397\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"
Sep 11 13:29:08.190887 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.190725    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lpj8\" (UniqueName: \"kubernetes.io/projected/8ca978df-3ba4-4a91-b512-81e1fdc7a397-kube-api-access-8lpj8\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp\" (UID: \"8ca978df-3ba4-4a91-b512-81e1fdc7a397\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"
Sep 11 13:29:08.190887 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.190747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ca978df-3ba4-4a91-b512-81e1fdc7a397-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp\" (UID: \"8ca978df-3ba4-4a91-b512-81e1fdc7a397\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"
Sep 11 13:29:08.190887 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.190790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glf4d\" (UniqueName: \"kubernetes.io/projected/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-kube-api-access-glf4d\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc\" (UID: \"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"
Sep 11 13:29:08.190887 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.190843    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc\" (UID: \"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"
Sep 11 13:29:08.191086 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.190893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc\" (UID: \"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"
Sep 11 13:29:08.191126 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.191090    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ca978df-3ba4-4a91-b512-81e1fdc7a397-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp\" (UID: \"8ca978df-3ba4-4a91-b512-81e1fdc7a397\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"
Sep 11 13:29:08.191181 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.191165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ca978df-3ba4-4a91-b512-81e1fdc7a397-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp\" (UID: \"8ca978df-3ba4-4a91-b512-81e1fdc7a397\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"
Sep 11 13:29:08.199979 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.199957    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lpj8\" (UniqueName: \"kubernetes.io/projected/8ca978df-3ba4-4a91-b512-81e1fdc7a397-kube-api-access-8lpj8\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp\" (UID: \"8ca978df-3ba4-4a91-b512-81e1fdc7a397\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"
Sep 11 13:29:08.238773 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.238696    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"]
Sep 11 13:29:08.241235 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.241217    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"
Sep 11 13:29:08.253064 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.253040    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"]
Sep 11 13:29:08.270399 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.270370    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"
Sep 11 13:29:08.292382 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.292353    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glf4d\" (UniqueName: \"kubernetes.io/projected/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-kube-api-access-glf4d\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc\" (UID: \"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"
Sep 11 13:29:08.292492 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.292398    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/201a1d57-63c5-48e5-9e5c-8da60255360b-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s\" (UID: \"201a1d57-63c5-48e5-9e5c-8da60255360b\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"
Sep 11 13:29:08.292492 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.292439    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc\" (UID: \"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"
Sep 11 13:29:08.292601 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.292487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc\" (UID: \"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"
Sep 11 13:29:08.292601 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.292517    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/201a1d57-63c5-48e5-9e5c-8da60255360b-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s\" (UID: \"201a1d57-63c5-48e5-9e5c-8da60255360b\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"
Sep 11 13:29:08.292703 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.292596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hw77\" (UniqueName: \"kubernetes.io/projected/201a1d57-63c5-48e5-9e5c-8da60255360b-kube-api-access-6hw77\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s\" (UID: \"201a1d57-63c5-48e5-9e5c-8da60255360b\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"
Sep 11 13:29:08.292871 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.292828    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc\" (UID: \"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"
Sep 11 13:29:08.292988 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.292961    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc\" (UID: \"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"
Sep 11 13:29:08.302541 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.302510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glf4d\" (UniqueName: \"kubernetes.io/projected/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-kube-api-access-glf4d\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc\" (UID: \"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"
Sep 11 13:29:08.355377 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.355342    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"
Sep 11 13:29:08.393255 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.393216    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20e683a1-0156-4625-86d2-c583d2a94d05-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz\" (UID: \"20e683a1-0156-4625-86d2-c583d2a94d05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"
Sep 11 13:29:08.393255 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.393256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hcl2\" (UniqueName: \"kubernetes.io/projected/20e683a1-0156-4625-86d2-c583d2a94d05-kube-api-access-5hcl2\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz\" (UID: \"20e683a1-0156-4625-86d2-c583d2a94d05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"
Sep 11 13:29:08.393501 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.393276    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20e683a1-0156-4625-86d2-c583d2a94d05-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz\" (UID: \"20e683a1-0156-4625-86d2-c583d2a94d05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"
Sep 11 13:29:08.393501 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.393349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hw77\" (UniqueName: \"kubernetes.io/projected/201a1d57-63c5-48e5-9e5c-8da60255360b-kube-api-access-6hw77\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s\" (UID: \"201a1d57-63c5-48e5-9e5c-8da60255360b\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"
Sep 11 13:29:08.393501 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.393405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/201a1d57-63c5-48e5-9e5c-8da60255360b-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s\" (UID: \"201a1d57-63c5-48e5-9e5c-8da60255360b\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"
Sep 11 13:29:08.393501 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.393451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/201a1d57-63c5-48e5-9e5c-8da60255360b-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s\" (UID: \"201a1d57-63c5-48e5-9e5c-8da60255360b\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"
Sep 11 13:29:08.393858 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.393835    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/201a1d57-63c5-48e5-9e5c-8da60255360b-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s\" (UID: \"201a1d57-63c5-48e5-9e5c-8da60255360b\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"
Sep 11 13:29:08.393961 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.393923    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/201a1d57-63c5-48e5-9e5c-8da60255360b-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s\" (UID: \"201a1d57-63c5-48e5-9e5c-8da60255360b\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"
Sep 11 13:29:08.404991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.404964    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hw77\" (UniqueName: \"kubernetes.io/projected/201a1d57-63c5-48e5-9e5c-8da60255360b-kube-api-access-6hw77\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s\" (UID: \"201a1d57-63c5-48e5-9e5c-8da60255360b\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"
Sep 11 13:29:08.443359 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.443325    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"
Sep 11 13:29:08.494507 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.494472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20e683a1-0156-4625-86d2-c583d2a94d05-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz\" (UID: \"20e683a1-0156-4625-86d2-c583d2a94d05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"
Sep 11 13:29:08.494651 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.494520    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hcl2\" (UniqueName: \"kubernetes.io/projected/20e683a1-0156-4625-86d2-c583d2a94d05-kube-api-access-5hcl2\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz\" (UID: \"20e683a1-0156-4625-86d2-c583d2a94d05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"
Sep 11 13:29:08.494772 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.494745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20e683a1-0156-4625-86d2-c583d2a94d05-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz\" (UID: \"20e683a1-0156-4625-86d2-c583d2a94d05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"
Sep 11 13:29:08.495479 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.494955    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20e683a1-0156-4625-86d2-c583d2a94d05-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz\" (UID: \"20e683a1-0156-4625-86d2-c583d2a94d05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"
Sep 11 13:29:08.495479 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.495153    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20e683a1-0156-4625-86d2-c583d2a94d05-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz\" (UID: \"20e683a1-0156-4625-86d2-c583d2a94d05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"
Sep 11 13:29:08.504441 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.504421    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hcl2\" (UniqueName: \"kubernetes.io/projected/20e683a1-0156-4625-86d2-c583d2a94d05-kube-api-access-5hcl2\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz\" (UID: \"20e683a1-0156-4625-86d2-c583d2a94d05\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"
Sep 11 13:29:08.550222 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.550193    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"
Sep 11 13:29:08.566599 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.566579    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"]
Sep 11 13:29:08.568529 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:29:08.568504    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ca978df_3ba4_4a91_b512_81e1fdc7a397.slice/crio-bfed0c16bfe4895d8592225ada92c23639cf31ae42713a3dd2cef37defe149bb WatchSource:0}: Error finding container bfed0c16bfe4895d8592225ada92c23639cf31ae42713a3dd2cef37defe149bb: Status 404 returned error can't find the container with id bfed0c16bfe4895d8592225ada92c23639cf31ae42713a3dd2cef37defe149bb
Sep 11 13:29:08.636764 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.636734    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"]
Sep 11 13:29:08.640123 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:29:08.640096    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e9571c4_3f36_4812_8fd4_a8ad738ccfd9.slice/crio-d902bcac3e91ea35c8a9fdc5880216e64e852c517143328ee032cb250533eb08 WatchSource:0}: Error finding container d902bcac3e91ea35c8a9fdc5880216e64e852c517143328ee032cb250533eb08: Status 404 returned error can't find the container with id d902bcac3e91ea35c8a9fdc5880216e64e852c517143328ee032cb250533eb08
Sep 11 13:29:08.741156 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.741127    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"]
Sep 11 13:29:08.763716 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:29:08.763687    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod201a1d57_63c5_48e5_9e5c_8da60255360b.slice/crio-04d12956f5acab8c32696441aa505c16f313d895257c26ecf1605d42929f1021 WatchSource:0}: Error finding container 04d12956f5acab8c32696441aa505c16f313d895257c26ecf1605d42929f1021: Status 404 returned error can't find the container with id 04d12956f5acab8c32696441aa505c16f313d895257c26ecf1605d42929f1021
Sep 11 13:29:08.828478 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:08.828445    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"]
Sep 11 13:29:08.828584 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:29:08.828514    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20e683a1_0156_4625_86d2_c583d2a94d05.slice/crio-bd08cef04a9ddf8db58e31b55ecc6921e0cf630050f9fd2064ab2b4a18c744ab WatchSource:0}: Error finding container bd08cef04a9ddf8db58e31b55ecc6921e0cf630050f9fd2064ab2b4a18c744ab: Status 404 returned error can't find the container with id bd08cef04a9ddf8db58e31b55ecc6921e0cf630050f9fd2064ab2b4a18c744ab
Sep 11 13:29:09.435359 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:09.435321    2580 generic.go:356] "Generic (PLEG): container finished" podID="20e683a1-0156-4625-86d2-c583d2a94d05" containerID="1f7fd75dfafe6467db0876d4efff12df0295332bbeedb574254a55378186d8ee" exitCode=0
Sep 11 13:29:09.435823 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:09.435411    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz" event={"ID":"20e683a1-0156-4625-86d2-c583d2a94d05","Type":"ContainerDied","Data":"1f7fd75dfafe6467db0876d4efff12df0295332bbeedb574254a55378186d8ee"}
Sep 11 13:29:09.435823 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:09.435456    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz" event={"ID":"20e683a1-0156-4625-86d2-c583d2a94d05","Type":"ContainerStarted","Data":"bd08cef04a9ddf8db58e31b55ecc6921e0cf630050f9fd2064ab2b4a18c744ab"}
Sep 11 13:29:09.437029 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:09.437007    2580 generic.go:356] "Generic (PLEG): container finished" podID="8ca978df-3ba4-4a91-b512-81e1fdc7a397" containerID="14c62d5a0f8ce3192a89b3a99fb66bc55253576d301609c172092c0e652e8ccd" exitCode=0
Sep 11 13:29:09.437125 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:09.437099    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp" event={"ID":"8ca978df-3ba4-4a91-b512-81e1fdc7a397","Type":"ContainerDied","Data":"14c62d5a0f8ce3192a89b3a99fb66bc55253576d301609c172092c0e652e8ccd"}
Sep 11 13:29:09.437189 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:09.437142    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp" event={"ID":"8ca978df-3ba4-4a91-b512-81e1fdc7a397","Type":"ContainerStarted","Data":"bfed0c16bfe4895d8592225ada92c23639cf31ae42713a3dd2cef37defe149bb"}
Sep 11 13:29:09.438598 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:09.438582    2580 generic.go:356] "Generic (PLEG): container finished" podID="9e9571c4-3f36-4812-8fd4-a8ad738ccfd9" containerID="86058722ccd81cb2167d3437b83bcd37880ba20d9e68677f994eec931f4d57bc" exitCode=0
Sep 11 13:29:09.438669 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:09.438652    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc" event={"ID":"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9","Type":"ContainerDied","Data":"86058722ccd81cb2167d3437b83bcd37880ba20d9e68677f994eec931f4d57bc"}
Sep 11 13:29:09.438716 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:09.438689    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc" event={"ID":"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9","Type":"ContainerStarted","Data":"d902bcac3e91ea35c8a9fdc5880216e64e852c517143328ee032cb250533eb08"}
Sep 11 13:29:09.440309 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:09.440287    2580 generic.go:356] "Generic (PLEG): container finished" podID="201a1d57-63c5-48e5-9e5c-8da60255360b" containerID="dc152227e27d21f513905365c6907404e184f5517a461c5a71e906c51a4bcbad" exitCode=0
Sep 11 13:29:09.440417 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:09.440340    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s" event={"ID":"201a1d57-63c5-48e5-9e5c-8da60255360b","Type":"ContainerDied","Data":"dc152227e27d21f513905365c6907404e184f5517a461c5a71e906c51a4bcbad"}
Sep 11 13:29:09.440417 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:09.440384    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s" event={"ID":"201a1d57-63c5-48e5-9e5c-8da60255360b","Type":"ContainerStarted","Data":"04d12956f5acab8c32696441aa505c16f313d895257c26ecf1605d42929f1021"}
Sep 11 13:29:10.447790 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:10.447707    2580 generic.go:356] "Generic (PLEG): container finished" podID="201a1d57-63c5-48e5-9e5c-8da60255360b" containerID="ae1bbcd7853675f0c735bfb44a533cb6a7d9dfd8478ecca921b4771387f7e7e2" exitCode=0
Sep 11 13:29:10.448263 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:10.447787    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s" event={"ID":"201a1d57-63c5-48e5-9e5c-8da60255360b","Type":"ContainerDied","Data":"ae1bbcd7853675f0c735bfb44a533cb6a7d9dfd8478ecca921b4771387f7e7e2"}
Sep 11 13:29:10.449737 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:10.449715    2580 generic.go:356] "Generic (PLEG): container finished" podID="20e683a1-0156-4625-86d2-c583d2a94d05" containerID="66cfeb1f1487000762ecfcdbdce317599070015b78d0b2b5d66ac1d484cb6ed9" exitCode=0
Sep 11 13:29:10.449833 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:10.449795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz" event={"ID":"20e683a1-0156-4625-86d2-c583d2a94d05","Type":"ContainerDied","Data":"66cfeb1f1487000762ecfcdbdce317599070015b78d0b2b5d66ac1d484cb6ed9"}
Sep 11 13:29:10.451719 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:10.451652    2580 generic.go:356] "Generic (PLEG): container finished" podID="8ca978df-3ba4-4a91-b512-81e1fdc7a397" containerID="a1b86a4243a0c5e73d0a20c4d75ab806edf5d2de2b23cbb66f59ac75268f93a3" exitCode=0
Sep 11 13:29:10.451790 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:10.451737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp" event={"ID":"8ca978df-3ba4-4a91-b512-81e1fdc7a397","Type":"ContainerDied","Data":"a1b86a4243a0c5e73d0a20c4d75ab806edf5d2de2b23cbb66f59ac75268f93a3"}
Sep 11 13:29:10.453684 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:10.453668    2580 generic.go:356] "Generic (PLEG): container finished" podID="9e9571c4-3f36-4812-8fd4-a8ad738ccfd9" containerID="b69039ae8e066bf09390a219d4d0ba07530ef1e5c4b79ee8da90cf46ba06637a" exitCode=0
Sep 11 13:29:10.453751 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:10.453696    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc" event={"ID":"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9","Type":"ContainerDied","Data":"b69039ae8e066bf09390a219d4d0ba07530ef1e5c4b79ee8da90cf46ba06637a"}
Sep 11 13:29:11.466521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:11.466482    2580 generic.go:356] "Generic (PLEG): container finished" podID="20e683a1-0156-4625-86d2-c583d2a94d05" containerID="e60faadb9d67b1877eea9333cfc8a70faf67d078772b63d74672590cfc54c1ac" exitCode=0
Sep 11 13:29:11.466957 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:11.466549    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz" event={"ID":"20e683a1-0156-4625-86d2-c583d2a94d05","Type":"ContainerDied","Data":"e60faadb9d67b1877eea9333cfc8a70faf67d078772b63d74672590cfc54c1ac"}
Sep 11 13:29:11.468666 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:11.468642    2580 generic.go:356] "Generic (PLEG): container finished" podID="8ca978df-3ba4-4a91-b512-81e1fdc7a397" containerID="3b593e8d3582e9518f59fce9cef17d2fd73a10054afc795b37565f3925014c83" exitCode=0
Sep 11 13:29:11.468768 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:11.468718    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp" event={"ID":"8ca978df-3ba4-4a91-b512-81e1fdc7a397","Type":"ContainerDied","Data":"3b593e8d3582e9518f59fce9cef17d2fd73a10054afc795b37565f3925014c83"}
Sep 11 13:29:11.470660 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:11.470636    2580 generic.go:356] "Generic (PLEG): container finished" podID="9e9571c4-3f36-4812-8fd4-a8ad738ccfd9" containerID="3fb5a0779283c67a61ba04b220693e79cb88c6a9b0b13f3c7b5eaccc33b4316e" exitCode=0
Sep 11 13:29:11.470747 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:11.470716    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc" event={"ID":"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9","Type":"ContainerDied","Data":"3fb5a0779283c67a61ba04b220693e79cb88c6a9b0b13f3c7b5eaccc33b4316e"}
Sep 11 13:29:11.472480 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:11.472462    2580 generic.go:356] "Generic (PLEG): container finished" podID="201a1d57-63c5-48e5-9e5c-8da60255360b" containerID="094e3222dfe746de5d4b31370ef1f96cc5d63dc51a554b6a8ad87f2377dfa00a" exitCode=0
Sep 11 13:29:11.472567 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:11.472488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s" event={"ID":"201a1d57-63c5-48e5-9e5c-8da60255360b","Type":"ContainerDied","Data":"094e3222dfe746de5d4b31370ef1f96cc5d63dc51a554b6a8ad87f2377dfa00a"}
Sep 11 13:29:12.607092 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.607053    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"
Sep 11 13:29:12.656658 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.656631    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"
Sep 11 13:29:12.670748 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.670715    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"
Sep 11 13:29:12.678217 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.678195    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"
Sep 11 13:29:12.730584 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.730504    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/201a1d57-63c5-48e5-9e5c-8da60255360b-util\") pod \"201a1d57-63c5-48e5-9e5c-8da60255360b\" (UID: \"201a1d57-63c5-48e5-9e5c-8da60255360b\") "
Sep 11 13:29:12.730584 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.730541    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/201a1d57-63c5-48e5-9e5c-8da60255360b-bundle\") pod \"201a1d57-63c5-48e5-9e5c-8da60255360b\" (UID: \"201a1d57-63c5-48e5-9e5c-8da60255360b\") "
Sep 11 13:29:12.730797 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.730623    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hw77\" (UniqueName: \"kubernetes.io/projected/201a1d57-63c5-48e5-9e5c-8da60255360b-kube-api-access-6hw77\") pod \"201a1d57-63c5-48e5-9e5c-8da60255360b\" (UID: \"201a1d57-63c5-48e5-9e5c-8da60255360b\") "
Sep 11 13:29:12.731402 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.731307    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/201a1d57-63c5-48e5-9e5c-8da60255360b-bundle" (OuterVolumeSpecName: "bundle") pod "201a1d57-63c5-48e5-9e5c-8da60255360b" (UID: "201a1d57-63c5-48e5-9e5c-8da60255360b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:12.732868 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.732843    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/201a1d57-63c5-48e5-9e5c-8da60255360b-kube-api-access-6hw77" (OuterVolumeSpecName: "kube-api-access-6hw77") pod "201a1d57-63c5-48e5-9e5c-8da60255360b" (UID: "201a1d57-63c5-48e5-9e5c-8da60255360b"). InnerVolumeSpecName "kube-api-access-6hw77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:29:12.735714 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.735690    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/201a1d57-63c5-48e5-9e5c-8da60255360b-util" (OuterVolumeSpecName: "util") pod "201a1d57-63c5-48e5-9e5c-8da60255360b" (UID: "201a1d57-63c5-48e5-9e5c-8da60255360b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:12.831259 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.831221    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ca978df-3ba4-4a91-b512-81e1fdc7a397-bundle\") pod \"8ca978df-3ba4-4a91-b512-81e1fdc7a397\" (UID: \"8ca978df-3ba4-4a91-b512-81e1fdc7a397\") "
Sep 11 13:29:12.831403 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.831271    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-util\") pod \"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9\" (UID: \"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9\") "
Sep 11 13:29:12.831403 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.831296    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-bundle\") pod \"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9\" (UID: \"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9\") "
Sep 11 13:29:12.831403 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.831333    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glf4d\" (UniqueName: \"kubernetes.io/projected/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-kube-api-access-glf4d\") pod \"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9\" (UID: \"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9\") "
Sep 11 13:29:12.831403 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.831361    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lpj8\" (UniqueName: \"kubernetes.io/projected/8ca978df-3ba4-4a91-b512-81e1fdc7a397-kube-api-access-8lpj8\") pod \"8ca978df-3ba4-4a91-b512-81e1fdc7a397\" (UID: \"8ca978df-3ba4-4a91-b512-81e1fdc7a397\") "
Sep 11 13:29:12.831403 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.831387    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hcl2\" (UniqueName: \"kubernetes.io/projected/20e683a1-0156-4625-86d2-c583d2a94d05-kube-api-access-5hcl2\") pod \"20e683a1-0156-4625-86d2-c583d2a94d05\" (UID: \"20e683a1-0156-4625-86d2-c583d2a94d05\") "
Sep 11 13:29:12.831669 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.831419    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20e683a1-0156-4625-86d2-c583d2a94d05-util\") pod \"20e683a1-0156-4625-86d2-c583d2a94d05\" (UID: \"20e683a1-0156-4625-86d2-c583d2a94d05\") "
Sep 11 13:29:12.831669 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.831467    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20e683a1-0156-4625-86d2-c583d2a94d05-bundle\") pod \"20e683a1-0156-4625-86d2-c583d2a94d05\" (UID: \"20e683a1-0156-4625-86d2-c583d2a94d05\") "
Sep 11 13:29:12.831669 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.831491    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ca978df-3ba4-4a91-b512-81e1fdc7a397-util\") pod \"8ca978df-3ba4-4a91-b512-81e1fdc7a397\" (UID: \"8ca978df-3ba4-4a91-b512-81e1fdc7a397\") "
Sep 11 13:29:12.831827 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.831737    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/201a1d57-63c5-48e5-9e5c-8da60255360b-bundle\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:12.831827 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.831755    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hw77\" (UniqueName: \"kubernetes.io/projected/201a1d57-63c5-48e5-9e5c-8da60255360b-kube-api-access-6hw77\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:12.831827 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.831770    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/201a1d57-63c5-48e5-9e5c-8da60255360b-util\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:12.832271 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.832236    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ca978df-3ba4-4a91-b512-81e1fdc7a397-bundle" (OuterVolumeSpecName: "bundle") pod "8ca978df-3ba4-4a91-b512-81e1fdc7a397" (UID: "8ca978df-3ba4-4a91-b512-81e1fdc7a397"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:12.832863 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.832795    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20e683a1-0156-4625-86d2-c583d2a94d05-bundle" (OuterVolumeSpecName: "bundle") pod "20e683a1-0156-4625-86d2-c583d2a94d05" (UID: "20e683a1-0156-4625-86d2-c583d2a94d05"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:12.832863 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.832822    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-bundle" (OuterVolumeSpecName: "bundle") pod "9e9571c4-3f36-4812-8fd4-a8ad738ccfd9" (UID: "9e9571c4-3f36-4812-8fd4-a8ad738ccfd9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:12.834139 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.834109    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ca978df-3ba4-4a91-b512-81e1fdc7a397-kube-api-access-8lpj8" (OuterVolumeSpecName: "kube-api-access-8lpj8") pod "8ca978df-3ba4-4a91-b512-81e1fdc7a397" (UID: "8ca978df-3ba4-4a91-b512-81e1fdc7a397"). InnerVolumeSpecName "kube-api-access-8lpj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:29:12.834251 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.834147    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-kube-api-access-glf4d" (OuterVolumeSpecName: "kube-api-access-glf4d") pod "9e9571c4-3f36-4812-8fd4-a8ad738ccfd9" (UID: "9e9571c4-3f36-4812-8fd4-a8ad738ccfd9"). InnerVolumeSpecName "kube-api-access-glf4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:29:12.835304 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.835283    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20e683a1-0156-4625-86d2-c583d2a94d05-kube-api-access-5hcl2" (OuterVolumeSpecName: "kube-api-access-5hcl2") pod "20e683a1-0156-4625-86d2-c583d2a94d05" (UID: "20e683a1-0156-4625-86d2-c583d2a94d05"). InnerVolumeSpecName "kube-api-access-5hcl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:29:12.838523 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.838497    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20e683a1-0156-4625-86d2-c583d2a94d05-util" (OuterVolumeSpecName: "util") pod "20e683a1-0156-4625-86d2-c583d2a94d05" (UID: "20e683a1-0156-4625-86d2-c583d2a94d05"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:12.840720 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.840695    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-util" (OuterVolumeSpecName: "util") pod "9e9571c4-3f36-4812-8fd4-a8ad738ccfd9" (UID: "9e9571c4-3f36-4812-8fd4-a8ad738ccfd9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:12.841088 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.841053    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ca978df-3ba4-4a91-b512-81e1fdc7a397-util" (OuterVolumeSpecName: "util") pod "8ca978df-3ba4-4a91-b512-81e1fdc7a397" (UID: "8ca978df-3ba4-4a91-b512-81e1fdc7a397"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:12.932947 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.932898    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glf4d\" (UniqueName: \"kubernetes.io/projected/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-kube-api-access-glf4d\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:12.932947 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.932945    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lpj8\" (UniqueName: \"kubernetes.io/projected/8ca978df-3ba4-4a91-b512-81e1fdc7a397-kube-api-access-8lpj8\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:12.932947 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.932954    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hcl2\" (UniqueName: \"kubernetes.io/projected/20e683a1-0156-4625-86d2-c583d2a94d05-kube-api-access-5hcl2\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:12.933187 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.932964    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20e683a1-0156-4625-86d2-c583d2a94d05-util\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:12.933187 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.932973    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20e683a1-0156-4625-86d2-c583d2a94d05-bundle\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:12.933187 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.932981    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ca978df-3ba4-4a91-b512-81e1fdc7a397-util\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:12.933187 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.932988    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ca978df-3ba4-4a91-b512-81e1fdc7a397-bundle\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:12.933187 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.932997    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-util\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:12.933187 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:12.933004    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9e9571c4-3f36-4812-8fd4-a8ad738ccfd9-bundle\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:13.484674 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:13.484647    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz"
Sep 11 13:29:13.484933 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:13.484643    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ct6sxz" event={"ID":"20e683a1-0156-4625-86d2-c583d2a94d05","Type":"ContainerDied","Data":"bd08cef04a9ddf8db58e31b55ecc6921e0cf630050f9fd2064ab2b4a18c744ab"}
Sep 11 13:29:13.484933 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:13.484758    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd08cef04a9ddf8db58e31b55ecc6921e0cf630050f9fd2064ab2b4a18c744ab"
Sep 11 13:29:13.486671 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:13.486636    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp"
Sep 11 13:29:13.486806 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:13.486670    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d74qzhp" event={"ID":"8ca978df-3ba4-4a91-b512-81e1fdc7a397","Type":"ContainerDied","Data":"bfed0c16bfe4895d8592225ada92c23639cf31ae42713a3dd2cef37defe149bb"}
Sep 11 13:29:13.486806 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:13.486713    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bfed0c16bfe4895d8592225ada92c23639cf31ae42713a3dd2cef37defe149bb"
Sep 11 13:29:13.488892 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:13.488861    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc"
Sep 11 13:29:13.489023 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:13.488892    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67hgdc" event={"ID":"9e9571c4-3f36-4812-8fd4-a8ad738ccfd9","Type":"ContainerDied","Data":"d902bcac3e91ea35c8a9fdc5880216e64e852c517143328ee032cb250533eb08"}
Sep 11 13:29:13.489023 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:13.488947    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d902bcac3e91ea35c8a9fdc5880216e64e852c517143328ee032cb250533eb08"
Sep 11 13:29:13.491049 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:13.491023    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s" event={"ID":"201a1d57-63c5-48e5-9e5c-8da60255360b","Type":"ContainerDied","Data":"04d12956f5acab8c32696441aa505c16f313d895257c26ecf1605d42929f1021"}
Sep 11 13:29:13.491049 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:13.491043    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atld7s"
Sep 11 13:29:13.491183 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:13.491066    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04d12956f5acab8c32696441aa505c16f313d895257c26ecf1605d42929f1021"
Sep 11 13:29:21.263780 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.263751    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-674c9cf75d-2m2g8"]
Sep 11 13:29:21.264165 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264117    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e9571c4-3f36-4812-8fd4-a8ad738ccfd9" containerName="util"
Sep 11 13:29:21.264165 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264130    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e9571c4-3f36-4812-8fd4-a8ad738ccfd9" containerName="util"
Sep 11 13:29:21.264165 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264142    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e9571c4-3f36-4812-8fd4-a8ad738ccfd9" containerName="pull"
Sep 11 13:29:21.264165 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264147    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e9571c4-3f36-4812-8fd4-a8ad738ccfd9" containerName="pull"
Sep 11 13:29:21.264165 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264155    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="201a1d57-63c5-48e5-9e5c-8da60255360b" containerName="util"
Sep 11 13:29:21.264165 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264161    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="201a1d57-63c5-48e5-9e5c-8da60255360b" containerName="util"
Sep 11 13:29:21.264165 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264168    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="201a1d57-63c5-48e5-9e5c-8da60255360b" containerName="pull"
Sep 11 13:29:21.264165 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264173    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="201a1d57-63c5-48e5-9e5c-8da60255360b" containerName="pull"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264180    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e683a1-0156-4625-86d2-c583d2a94d05" containerName="pull"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264188    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e683a1-0156-4625-86d2-c583d2a94d05" containerName="pull"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264197    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ca978df-3ba4-4a91-b512-81e1fdc7a397" containerName="extract"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264207    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ca978df-3ba4-4a91-b512-81e1fdc7a397" containerName="extract"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264222    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e683a1-0156-4625-86d2-c583d2a94d05" containerName="extract"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264227    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e683a1-0156-4625-86d2-c583d2a94d05" containerName="extract"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264236    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e9571c4-3f36-4812-8fd4-a8ad738ccfd9" containerName="extract"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264242    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e9571c4-3f36-4812-8fd4-a8ad738ccfd9" containerName="extract"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264251    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ca978df-3ba4-4a91-b512-81e1fdc7a397" containerName="util"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264258    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ca978df-3ba4-4a91-b512-81e1fdc7a397" containerName="util"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264269    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e683a1-0156-4625-86d2-c583d2a94d05" containerName="util"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264277    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e683a1-0156-4625-86d2-c583d2a94d05" containerName="util"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264284    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ca978df-3ba4-4a91-b512-81e1fdc7a397" containerName="pull"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264289    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ca978df-3ba4-4a91-b512-81e1fdc7a397" containerName="pull"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264295    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="201a1d57-63c5-48e5-9e5c-8da60255360b" containerName="extract"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264303    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="201a1d57-63c5-48e5-9e5c-8da60255360b" containerName="extract"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264364    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="201a1d57-63c5-48e5-9e5c-8da60255360b" containerName="extract"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264373    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e9571c4-3f36-4812-8fd4-a8ad738ccfd9" containerName="extract"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264381    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e683a1-0156-4625-86d2-c583d2a94d05" containerName="extract"
Sep 11 13:29:21.264451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.264388    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ca978df-3ba4-4a91-b512-81e1fdc7a397" containerName="extract"
Sep 11 13:29:21.266320 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.266285    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.290198 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.290169    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-674c9cf75d-2m2g8"]
Sep 11 13:29:21.307886 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.307848    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-console-config\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.308049 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.307886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-console-serving-cert\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.308049 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.307986    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmpcz\" (UniqueName: \"kubernetes.io/projected/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-kube-api-access-zmpcz\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.308049 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.308033    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-oauth-serving-cert\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.308179 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.308081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-service-ca\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.308179 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.308100    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-console-oauth-config\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.308179 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.308121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-trusted-ca-bundle\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.408655 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.408624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmpcz\" (UniqueName: \"kubernetes.io/projected/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-kube-api-access-zmpcz\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.408833 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.408667    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-oauth-serving-cert\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.408833 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.408816    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-service-ca\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.409012 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.408848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-console-oauth-config\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.409012 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.408870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-trusted-ca-bundle\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.409012 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.408950    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-console-config\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.409012 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.408974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-console-serving-cert\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.409577 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.409552    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-oauth-serving-cert\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.409710 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.409681    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-console-config\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.409834 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.409687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-trusted-ca-bundle\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.409834 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.409786    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-service-ca\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.411417 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.411390    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-console-serving-cert\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.411503 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.411421    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-console-oauth-config\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.433211 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.433185    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmpcz\" (UniqueName: \"kubernetes.io/projected/bd3a03cb-d6fe-470c-b166-57c3d1a9e49d-kube-api-access-zmpcz\") pod \"console-674c9cf75d-2m2g8\" (UID: \"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d\") " pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.575530 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.575504    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:21.887614 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:29:21.887587    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd3a03cb_d6fe_470c_b166_57c3d1a9e49d.slice/crio-82045a51c2cd6191c894ceb9a043be57f22efdafd729e91362b10d1aa68cc788 WatchSource:0}: Error finding container 82045a51c2cd6191c894ceb9a043be57f22efdafd729e91362b10d1aa68cc788: Status 404 returned error can't find the container with id 82045a51c2cd6191c894ceb9a043be57f22efdafd729e91362b10d1aa68cc788
Sep 11 13:29:21.893336 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:21.893314    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-674c9cf75d-2m2g8"]
Sep 11 13:29:22.543542 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:22.543504    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-674c9cf75d-2m2g8" event={"ID":"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d","Type":"ContainerStarted","Data":"c06db10a63c16fdccfe372ea7662235675e984d7986f2304914b360b4a39e1f7"}
Sep 11 13:29:22.543976 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:22.543556    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-674c9cf75d-2m2g8" event={"ID":"bd3a03cb-d6fe-470c-b166-57c3d1a9e49d","Type":"ContainerStarted","Data":"82045a51c2cd6191c894ceb9a043be57f22efdafd729e91362b10d1aa68cc788"}
Sep 11 13:29:31.575863 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:31.575824    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:31.576357 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:31.575878    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:31.581312 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:31.581281    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:31.596933 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:31.592327    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-674c9cf75d-2m2g8"
Sep 11 13:29:31.618219 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:31.618166    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-674c9cf75d-2m2g8" podStartSLOduration=10.618150028 podStartE2EDuration="10.618150028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:29:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:29:22.587819693 +0000 UTC m=+481.806357185" watchObservedRunningTime="2026-09-11 13:29:31.618150028 +0000 UTC m=+490.836687523"
Sep 11 13:29:31.735414 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:31.735379    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d748c9f7c-rvm8z"]
Sep 11 13:29:32.716050 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.716019    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg"]
Sep 11 13:29:32.718447 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.718429    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg"
Sep 11 13:29:32.725215 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.725188    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 13:29:32.725368 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.725259    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 11 13:29:32.725368 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.725274    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-whbpf\""
Sep 11 13:29:32.726045 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.726022    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 11 13:29:32.726176 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.726095    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 13:29:32.746588 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.746566    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg"]
Sep 11 13:29:32.805442 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.805410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj5mh\" (UniqueName: \"kubernetes.io/projected/750c0e85-0552-4e45-82aa-ee81c9b6215e-kube-api-access-gj5mh\") pod \"kuadrant-console-plugin-fbdd4f445-qd7cg\" (UID: \"750c0e85-0552-4e45-82aa-ee81c9b6215e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg"
Sep 11 13:29:32.805641 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.805459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/750c0e85-0552-4e45-82aa-ee81c9b6215e-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-qd7cg\" (UID: \"750c0e85-0552-4e45-82aa-ee81c9b6215e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg"
Sep 11 13:29:32.805641 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.805540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/750c0e85-0552-4e45-82aa-ee81c9b6215e-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-qd7cg\" (UID: \"750c0e85-0552-4e45-82aa-ee81c9b6215e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg"
Sep 11 13:29:32.907044 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.907006    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj5mh\" (UniqueName: \"kubernetes.io/projected/750c0e85-0552-4e45-82aa-ee81c9b6215e-kube-api-access-gj5mh\") pod \"kuadrant-console-plugin-fbdd4f445-qd7cg\" (UID: \"750c0e85-0552-4e45-82aa-ee81c9b6215e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg"
Sep 11 13:29:32.907249 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.907055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/750c0e85-0552-4e45-82aa-ee81c9b6215e-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-qd7cg\" (UID: \"750c0e85-0552-4e45-82aa-ee81c9b6215e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg"
Sep 11 13:29:32.907249 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.907090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/750c0e85-0552-4e45-82aa-ee81c9b6215e-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-qd7cg\" (UID: \"750c0e85-0552-4e45-82aa-ee81c9b6215e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg"
Sep 11 13:29:32.907728 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.907707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/750c0e85-0552-4e45-82aa-ee81c9b6215e-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-qd7cg\" (UID: \"750c0e85-0552-4e45-82aa-ee81c9b6215e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg"
Sep 11 13:29:32.909492 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.909476    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/750c0e85-0552-4e45-82aa-ee81c9b6215e-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-qd7cg\" (UID: \"750c0e85-0552-4e45-82aa-ee81c9b6215e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg"
Sep 11 13:29:32.919651 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:32.919626    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj5mh\" (UniqueName: \"kubernetes.io/projected/750c0e85-0552-4e45-82aa-ee81c9b6215e-kube-api-access-gj5mh\") pod \"kuadrant-console-plugin-fbdd4f445-qd7cg\" (UID: \"750c0e85-0552-4e45-82aa-ee81c9b6215e\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg"
Sep 11 13:29:33.028591 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:33.028488    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg"
Sep 11 13:29:33.332341 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:33.332315    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg"]
Sep 11 13:29:33.332582 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:29:33.332559    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod750c0e85_0552_4e45_82aa_ee81c9b6215e.slice/crio-40ccc167668e62dfa96844d3184d62d4b4652ca24dfb12b2af5ec7a87e86099e WatchSource:0}: Error finding container 40ccc167668e62dfa96844d3184d62d4b4652ca24dfb12b2af5ec7a87e86099e: Status 404 returned error can't find the container with id 40ccc167668e62dfa96844d3184d62d4b4652ca24dfb12b2af5ec7a87e86099e
Sep 11 13:29:33.599551 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:33.599457    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg" event={"ID":"750c0e85-0552-4e45-82aa-ee81c9b6215e","Type":"ContainerStarted","Data":"40ccc167668e62dfa96844d3184d62d4b4652ca24dfb12b2af5ec7a87e86099e"}
Sep 11 13:29:37.627165 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:37.627124    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg" event={"ID":"750c0e85-0552-4e45-82aa-ee81c9b6215e","Type":"ContainerStarted","Data":"7e805e5d85e036327ba48e7542f5463e34257b51bb2a8d3fd40c182063ad404b"}
Sep 11 13:29:37.651517 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:37.651469    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-qd7cg" podStartSLOduration=1.571633075 podStartE2EDuration="5.651454474s" totalImagesPullingTime="4.079821399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:29:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:29:37.649364906 +0000 UTC m=+496.867902399" watchObservedRunningTime="2026-09-11 13:29:37.651454474 +0000 UTC m=+496.869991967"
Sep 11 13:29:56.754232 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:56.754192    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d748c9f7c-rvm8z" podUID="48a9fd55-064f-4509-aad3-0e21020f8502" containerName="console" containerID="cri-o://7420eb2ae3fe06e7244d97ab4bbe54d47f1626a94eee71e04a0a80dbe7c07d3b" gracePeriod=15
Sep 11 13:29:56.999219 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:56.999191    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:29:57.122807 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.122773    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-trusted-ca-bundle\") pod \"48a9fd55-064f-4509-aad3-0e21020f8502\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") "
Sep 11 13:29:57.123009 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.122822    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-service-ca\") pod \"48a9fd55-064f-4509-aad3-0e21020f8502\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") "
Sep 11 13:29:57.123009 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.122846    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-console-config\") pod \"48a9fd55-064f-4509-aad3-0e21020f8502\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") "
Sep 11 13:29:57.123009 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.122893    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jbkr\" (UniqueName: \"kubernetes.io/projected/48a9fd55-064f-4509-aad3-0e21020f8502-kube-api-access-5jbkr\") pod \"48a9fd55-064f-4509-aad3-0e21020f8502\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") "
Sep 11 13:29:57.123009 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.122945    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/48a9fd55-064f-4509-aad3-0e21020f8502-console-serving-cert\") pod \"48a9fd55-064f-4509-aad3-0e21020f8502\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") "
Sep 11 13:29:57.123009 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.122961    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-oauth-serving-cert\") pod \"48a9fd55-064f-4509-aad3-0e21020f8502\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") "
Sep 11 13:29:57.123009 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.123002    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/48a9fd55-064f-4509-aad3-0e21020f8502-console-oauth-config\") pod \"48a9fd55-064f-4509-aad3-0e21020f8502\" (UID: \"48a9fd55-064f-4509-aad3-0e21020f8502\") "
Sep 11 13:29:57.123348 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.123238    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-service-ca" (OuterVolumeSpecName: "service-ca") pod "48a9fd55-064f-4509-aad3-0e21020f8502" (UID: "48a9fd55-064f-4509-aad3-0e21020f8502"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:29:57.123348 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.123247    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "48a9fd55-064f-4509-aad3-0e21020f8502" (UID: "48a9fd55-064f-4509-aad3-0e21020f8502"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:29:57.123348 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.123313    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-console-config" (OuterVolumeSpecName: "console-config") pod "48a9fd55-064f-4509-aad3-0e21020f8502" (UID: "48a9fd55-064f-4509-aad3-0e21020f8502"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:29:57.123593 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.123573    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "48a9fd55-064f-4509-aad3-0e21020f8502" (UID: "48a9fd55-064f-4509-aad3-0e21020f8502"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:29:57.125151 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.125128    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48a9fd55-064f-4509-aad3-0e21020f8502-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "48a9fd55-064f-4509-aad3-0e21020f8502" (UID: "48a9fd55-064f-4509-aad3-0e21020f8502"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:29:57.125567 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.125542    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48a9fd55-064f-4509-aad3-0e21020f8502-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "48a9fd55-064f-4509-aad3-0e21020f8502" (UID: "48a9fd55-064f-4509-aad3-0e21020f8502"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:29:57.125567 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.125554    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48a9fd55-064f-4509-aad3-0e21020f8502-kube-api-access-5jbkr" (OuterVolumeSpecName: "kube-api-access-5jbkr") pod "48a9fd55-064f-4509-aad3-0e21020f8502" (UID: "48a9fd55-064f-4509-aad3-0e21020f8502"). InnerVolumeSpecName "kube-api-access-5jbkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:29:57.224127 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.224087    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jbkr\" (UniqueName: \"kubernetes.io/projected/48a9fd55-064f-4509-aad3-0e21020f8502-kube-api-access-5jbkr\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:57.224127 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.224118    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/48a9fd55-064f-4509-aad3-0e21020f8502-console-serving-cert\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:57.224127 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.224127    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-oauth-serving-cert\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:57.224127 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.224135    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/48a9fd55-064f-4509-aad3-0e21020f8502-console-oauth-config\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:57.224420 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.224144    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-trusted-ca-bundle\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:57.224420 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.224153    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-service-ca\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:57.224420 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.224163    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/48a9fd55-064f-4509-aad3-0e21020f8502-console-config\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:29:57.728607 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.728575    2580 generic.go:356] "Generic (PLEG): container finished" podID="48a9fd55-064f-4509-aad3-0e21020f8502" containerID="7420eb2ae3fe06e7244d97ab4bbe54d47f1626a94eee71e04a0a80dbe7c07d3b" exitCode=0
Sep 11 13:29:57.728799 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.728632    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d748c9f7c-rvm8z"
Sep 11 13:29:57.728799 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.728667    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d748c9f7c-rvm8z" event={"ID":"48a9fd55-064f-4509-aad3-0e21020f8502","Type":"ContainerDied","Data":"7420eb2ae3fe06e7244d97ab4bbe54d47f1626a94eee71e04a0a80dbe7c07d3b"}
Sep 11 13:29:57.728799 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.728718    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d748c9f7c-rvm8z" event={"ID":"48a9fd55-064f-4509-aad3-0e21020f8502","Type":"ContainerDied","Data":"3a9a3a56da64ad3e35962b268e985ed0d7159ea6ab2b3bd4f68c57e5560acf2b"}
Sep 11 13:29:57.728799 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.728737    2580 scope.go:117] "RemoveContainer" containerID="7420eb2ae3fe06e7244d97ab4bbe54d47f1626a94eee71e04a0a80dbe7c07d3b"
Sep 11 13:29:57.737754 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.737738    2580 scope.go:117] "RemoveContainer" containerID="7420eb2ae3fe06e7244d97ab4bbe54d47f1626a94eee71e04a0a80dbe7c07d3b"
Sep 11 13:29:57.738032 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:29:57.738013    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7420eb2ae3fe06e7244d97ab4bbe54d47f1626a94eee71e04a0a80dbe7c07d3b\": container with ID starting with 7420eb2ae3fe06e7244d97ab4bbe54d47f1626a94eee71e04a0a80dbe7c07d3b not found: ID does not exist" containerID="7420eb2ae3fe06e7244d97ab4bbe54d47f1626a94eee71e04a0a80dbe7c07d3b"
Sep 11 13:29:57.738083 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.738041    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7420eb2ae3fe06e7244d97ab4bbe54d47f1626a94eee71e04a0a80dbe7c07d3b"} err="failed to get container status \"7420eb2ae3fe06e7244d97ab4bbe54d47f1626a94eee71e04a0a80dbe7c07d3b\": rpc error: code = NotFound desc = could not find container \"7420eb2ae3fe06e7244d97ab4bbe54d47f1626a94eee71e04a0a80dbe7c07d3b\": container with ID starting with 7420eb2ae3fe06e7244d97ab4bbe54d47f1626a94eee71e04a0a80dbe7c07d3b not found: ID does not exist"
Sep 11 13:29:57.750020 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.749995    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d748c9f7c-rvm8z"]
Sep 11 13:29:57.756744 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:57.756723    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d748c9f7c-rvm8z"]
Sep 11 13:29:59.314988 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:29:59.314956    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48a9fd55-064f-4509-aad3-0e21020f8502" path="/var/lib/kubelet/pods/48a9fd55-064f-4509-aad3-0e21020f8502/volumes"
Sep 11 13:30:28.633873 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:28.633838    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:30:28.634326 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:28.634219    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48a9fd55-064f-4509-aad3-0e21020f8502" containerName="console"
Sep 11 13:30:28.634326 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:28.634231    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="48a9fd55-064f-4509-aad3-0e21020f8502" containerName="console"
Sep 11 13:30:28.634326 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:28.634299    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="48a9fd55-064f-4509-aad3-0e21020f8502" containerName="console"
Sep 11 13:30:28.638375 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:28.638358    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"
Sep 11 13:30:28.645203 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:28.645178    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 13:30:28.659230 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:28.659198    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:30:28.694245 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:28.694213    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3d4fcc93-924f-439a-9eec-967c670a886e-config-file\") pod \"limitador-limitador-84994f7ffc-kbbvr\" (UID: \"3d4fcc93-924f-439a-9eec-967c670a886e\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"
Sep 11 13:30:28.694365 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:28.694274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvw4w\" (UniqueName: \"kubernetes.io/projected/3d4fcc93-924f-439a-9eec-967c670a886e-kube-api-access-zvw4w\") pod \"limitador-limitador-84994f7ffc-kbbvr\" (UID: \"3d4fcc93-924f-439a-9eec-967c670a886e\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"
Sep 11 13:30:28.708227 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:28.708196    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:30:28.795122 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:28.795084    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3d4fcc93-924f-439a-9eec-967c670a886e-config-file\") pod \"limitador-limitador-84994f7ffc-kbbvr\" (UID: \"3d4fcc93-924f-439a-9eec-967c670a886e\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"
Sep 11 13:30:28.795296 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:28.795148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvw4w\" (UniqueName: \"kubernetes.io/projected/3d4fcc93-924f-439a-9eec-967c670a886e-kube-api-access-zvw4w\") pod \"limitador-limitador-84994f7ffc-kbbvr\" (UID: \"3d4fcc93-924f-439a-9eec-967c670a886e\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"
Sep 11 13:30:28.795721 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:28.795700    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3d4fcc93-924f-439a-9eec-967c670a886e-config-file\") pod \"limitador-limitador-84994f7ffc-kbbvr\" (UID: \"3d4fcc93-924f-439a-9eec-967c670a886e\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"
Sep 11 13:30:28.806000 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:28.805976    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvw4w\" (UniqueName: \"kubernetes.io/projected/3d4fcc93-924f-439a-9eec-967c670a886e-kube-api-access-zvw4w\") pod \"limitador-limitador-84994f7ffc-kbbvr\" (UID: \"3d4fcc93-924f-439a-9eec-967c670a886e\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"
Sep 11 13:30:28.948550 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:28.948459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"
Sep 11 13:30:29.116961 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:29.116925    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-hdxf5"]
Sep 11 13:30:29.120476 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:29.120460    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-hdxf5"
Sep 11 13:30:29.125330 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:29.125309    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rq57j\""
Sep 11 13:30:29.136633 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:29.136606    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-hdxf5"]
Sep 11 13:30:29.200862 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:29.200788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxl2n\" (UniqueName: \"kubernetes.io/projected/962f1ba5-e58b-4559-8e5b-9c1252413147-kube-api-access-dxl2n\") pod \"authorino-7bcfd8f9cd-hdxf5\" (UID: \"962f1ba5-e58b-4559-8e5b-9c1252413147\") " pod="kuadrant-system/authorino-7bcfd8f9cd-hdxf5"
Sep 11 13:30:29.227164 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:29.227139    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:30:29.230045 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:30:29.230012    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d4fcc93_924f_439a_9eec_967c670a886e.slice/crio-7c105705bffd8c04db963d65ff9e24dbb095c164df593b9cecb882232cc315de WatchSource:0}: Error finding container 7c105705bffd8c04db963d65ff9e24dbb095c164df593b9cecb882232cc315de: Status 404 returned error can't find the container with id 7c105705bffd8c04db963d65ff9e24dbb095c164df593b9cecb882232cc315de
Sep 11 13:30:29.302170 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:29.302131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxl2n\" (UniqueName: \"kubernetes.io/projected/962f1ba5-e58b-4559-8e5b-9c1252413147-kube-api-access-dxl2n\") pod \"authorino-7bcfd8f9cd-hdxf5\" (UID: \"962f1ba5-e58b-4559-8e5b-9c1252413147\") " pod="kuadrant-system/authorino-7bcfd8f9cd-hdxf5"
Sep 11 13:30:29.312074 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:29.312050    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxl2n\" (UniqueName: \"kubernetes.io/projected/962f1ba5-e58b-4559-8e5b-9c1252413147-kube-api-access-dxl2n\") pod \"authorino-7bcfd8f9cd-hdxf5\" (UID: \"962f1ba5-e58b-4559-8e5b-9c1252413147\") " pod="kuadrant-system/authorino-7bcfd8f9cd-hdxf5"
Sep 11 13:30:29.429458 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:29.429428    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-hdxf5"
Sep 11 13:30:29.701116 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:29.701081    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-hdxf5"]
Sep 11 13:30:29.702748 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:30:29.702719    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod962f1ba5_e58b_4559_8e5b_9c1252413147.slice/crio-562257ce350a3f3fc291f66828daf829cf4f726fcbf30eb8d1f5076fee3c497f WatchSource:0}: Error finding container 562257ce350a3f3fc291f66828daf829cf4f726fcbf30eb8d1f5076fee3c497f: Status 404 returned error can't find the container with id 562257ce350a3f3fc291f66828daf829cf4f726fcbf30eb8d1f5076fee3c497f
Sep 11 13:30:29.893725 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:29.893691    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-kbbvr" event={"ID":"3d4fcc93-924f-439a-9eec-967c670a886e","Type":"ContainerStarted","Data":"7c105705bffd8c04db963d65ff9e24dbb095c164df593b9cecb882232cc315de"}
Sep 11 13:30:29.894791 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:29.894765    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-hdxf5" event={"ID":"962f1ba5-e58b-4559-8e5b-9c1252413147","Type":"ContainerStarted","Data":"562257ce350a3f3fc291f66828daf829cf4f726fcbf30eb8d1f5076fee3c497f"}
Sep 11 13:30:31.907579 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:31.907538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-hdxf5" event={"ID":"962f1ba5-e58b-4559-8e5b-9c1252413147","Type":"ContainerStarted","Data":"edb2678eb133ebe0301337f74e00d3bb302de028963c40a23ec64c06491d07e6"}
Sep 11 13:30:31.926031 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:31.925964    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-hdxf5" podStartSLOduration=0.78965713 podStartE2EDuration="2.925947381s" totalImagesPullingTime="2.136290251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:30:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:30:31.923529344 +0000 UTC m=+551.142066838" watchObservedRunningTime="2026-09-11 13:30:31.925947381 +0000 UTC m=+551.144484870"
Sep 11 13:30:32.135017 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:32.134936    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-hdxf5"]
Sep 11 13:30:33.918448 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:33.918381    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-hdxf5" podUID="962f1ba5-e58b-4559-8e5b-9c1252413147" containerName="authorino" containerID="cri-o://edb2678eb133ebe0301337f74e00d3bb302de028963c40a23ec64c06491d07e6" gracePeriod=30
Sep 11 13:30:34.157931 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:34.157895    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-hdxf5"
Sep 11 13:30:34.245999 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:34.245852    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxl2n\" (UniqueName: \"kubernetes.io/projected/962f1ba5-e58b-4559-8e5b-9c1252413147-kube-api-access-dxl2n\") pod \"962f1ba5-e58b-4559-8e5b-9c1252413147\" (UID: \"962f1ba5-e58b-4559-8e5b-9c1252413147\") "
Sep 11 13:30:34.248043 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:34.248019    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/962f1ba5-e58b-4559-8e5b-9c1252413147-kube-api-access-dxl2n" (OuterVolumeSpecName: "kube-api-access-dxl2n") pod "962f1ba5-e58b-4559-8e5b-9c1252413147" (UID: "962f1ba5-e58b-4559-8e5b-9c1252413147"). InnerVolumeSpecName "kube-api-access-dxl2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:30:34.347448 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:34.347416    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxl2n\" (UniqueName: \"kubernetes.io/projected/962f1ba5-e58b-4559-8e5b-9c1252413147-kube-api-access-dxl2n\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:30:34.924021 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:34.923985    2580 generic.go:356] "Generic (PLEG): container finished" podID="962f1ba5-e58b-4559-8e5b-9c1252413147" containerID="edb2678eb133ebe0301337f74e00d3bb302de028963c40a23ec64c06491d07e6" exitCode=0
Sep 11 13:30:34.924439 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:34.924032    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-hdxf5"
Sep 11 13:30:34.924439 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:34.924074    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-hdxf5" event={"ID":"962f1ba5-e58b-4559-8e5b-9c1252413147","Type":"ContainerDied","Data":"edb2678eb133ebe0301337f74e00d3bb302de028963c40a23ec64c06491d07e6"}
Sep 11 13:30:34.924439 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:34.924114    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-hdxf5" event={"ID":"962f1ba5-e58b-4559-8e5b-9c1252413147","Type":"ContainerDied","Data":"562257ce350a3f3fc291f66828daf829cf4f726fcbf30eb8d1f5076fee3c497f"}
Sep 11 13:30:34.924439 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:34.924130    2580 scope.go:117] "RemoveContainer" containerID="edb2678eb133ebe0301337f74e00d3bb302de028963c40a23ec64c06491d07e6"
Sep 11 13:30:34.933650 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:34.933636    2580 scope.go:117] "RemoveContainer" containerID="edb2678eb133ebe0301337f74e00d3bb302de028963c40a23ec64c06491d07e6"
Sep 11 13:30:34.933873 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:30:34.933857    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edb2678eb133ebe0301337f74e00d3bb302de028963c40a23ec64c06491d07e6\": container with ID starting with edb2678eb133ebe0301337f74e00d3bb302de028963c40a23ec64c06491d07e6 not found: ID does not exist" containerID="edb2678eb133ebe0301337f74e00d3bb302de028963c40a23ec64c06491d07e6"
Sep 11 13:30:34.933933 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:34.933880    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb2678eb133ebe0301337f74e00d3bb302de028963c40a23ec64c06491d07e6"} err="failed to get container status \"edb2678eb133ebe0301337f74e00d3bb302de028963c40a23ec64c06491d07e6\": rpc error: code = NotFound desc = could not find container \"edb2678eb133ebe0301337f74e00d3bb302de028963c40a23ec64c06491d07e6\": container with ID starting with edb2678eb133ebe0301337f74e00d3bb302de028963c40a23ec64c06491d07e6 not found: ID does not exist"
Sep 11 13:30:34.947388 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:34.947363    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-hdxf5"]
Sep 11 13:30:34.953138 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:34.953119    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-hdxf5"]
Sep 11 13:30:35.315489 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:35.315460    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="962f1ba5-e58b-4559-8e5b-9c1252413147" path="/var/lib/kubelet/pods/962f1ba5-e58b-4559-8e5b-9c1252413147/volumes"
Sep 11 13:30:45.981428 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:45.981388    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-kbbvr" event={"ID":"3d4fcc93-924f-439a-9eec-967c670a886e","Type":"ContainerStarted","Data":"b9e94c21b8bccf63c361d4a3aba9164603cd175687b4a1c61748b52ce6168eeb"}
Sep 11 13:30:45.981818 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:45.981502    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"
Sep 11 13:30:46.011992 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:46.011885    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-kbbvr" podStartSLOduration=1.488686491 podStartE2EDuration="18.011871228s" totalImagesPullingTime="16.523184737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:30:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:30:46.011370227 +0000 UTC m=+565.229907720" watchObservedRunningTime="2026-09-11 13:30:46.011871228 +0000 UTC m=+565.230408720"
Sep 11 13:30:56.988951 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:30:56.988887    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"
Sep 11 13:31:12.977549 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:12.977514    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-bpn65"]
Sep 11 13:31:12.978006 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:12.977845    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="962f1ba5-e58b-4559-8e5b-9c1252413147" containerName="authorino"
Sep 11 13:31:12.978006 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:12.977856    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="962f1ba5-e58b-4559-8e5b-9c1252413147" containerName="authorino"
Sep 11 13:31:12.978006 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:12.977927    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="962f1ba5-e58b-4559-8e5b-9c1252413147" containerName="authorino"
Sep 11 13:31:12.982888 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:12.982869    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-bpn65"
Sep 11 13:31:12.985619 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:12.985580    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rq57j\""
Sep 11 13:31:12.991325 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:12.991306    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-bpn65"]
Sep 11 13:31:13.070498 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.070460    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq7fd\" (UniqueName: \"kubernetes.io/projected/c990e6c6-2418-4bb8-ac6b-3b061681f87c-kube-api-access-pq7fd\") pod \"authorino-557f876f69-bpn65\" (UID: \"c990e6c6-2418-4bb8-ac6b-3b061681f87c\") " pod="kuadrant-system/authorino-557f876f69-bpn65"
Sep 11 13:31:13.158840 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.158801    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-bpn65"]
Sep 11 13:31:13.159113 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:31:13.159094    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-pq7fd], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-bpn65" podUID="c990e6c6-2418-4bb8-ac6b-3b061681f87c"
Sep 11 13:31:13.171263 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.171234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq7fd\" (UniqueName: \"kubernetes.io/projected/c990e6c6-2418-4bb8-ac6b-3b061681f87c-kube-api-access-pq7fd\") pod \"authorino-557f876f69-bpn65\" (UID: \"c990e6c6-2418-4bb8-ac6b-3b061681f87c\") " pod="kuadrant-system/authorino-557f876f69-bpn65"
Sep 11 13:31:13.182352 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.182328    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq7fd\" (UniqueName: \"kubernetes.io/projected/c990e6c6-2418-4bb8-ac6b-3b061681f87c-kube-api-access-pq7fd\") pod \"authorino-557f876f69-bpn65\" (UID: \"c990e6c6-2418-4bb8-ac6b-3b061681f87c\") " pod="kuadrant-system/authorino-557f876f69-bpn65"
Sep 11 13:31:13.335641 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.335610    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-774f46b7b9-pd6zf"]
Sep 11 13:31:13.338512 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.338495    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-774f46b7b9-pd6zf"
Sep 11 13:31:13.342049 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.342026    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 13:31:13.349642 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.349620    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-774f46b7b9-pd6zf"]
Sep 11 13:31:13.372223 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.372194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6688a0d7-bdcc-42d5-9b5c-4478b7b609b6-tls-cert\") pod \"authorino-774f46b7b9-pd6zf\" (UID: \"6688a0d7-bdcc-42d5-9b5c-4478b7b609b6\") " pod="kuadrant-system/authorino-774f46b7b9-pd6zf"
Sep 11 13:31:13.372357 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.372261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqkcs\" (UniqueName: \"kubernetes.io/projected/6688a0d7-bdcc-42d5-9b5c-4478b7b609b6-kube-api-access-pqkcs\") pod \"authorino-774f46b7b9-pd6zf\" (UID: \"6688a0d7-bdcc-42d5-9b5c-4478b7b609b6\") " pod="kuadrant-system/authorino-774f46b7b9-pd6zf"
Sep 11 13:31:13.473549 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.473516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6688a0d7-bdcc-42d5-9b5c-4478b7b609b6-tls-cert\") pod \"authorino-774f46b7b9-pd6zf\" (UID: \"6688a0d7-bdcc-42d5-9b5c-4478b7b609b6\") " pod="kuadrant-system/authorino-774f46b7b9-pd6zf"
Sep 11 13:31:13.473549 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.473558    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqkcs\" (UniqueName: \"kubernetes.io/projected/6688a0d7-bdcc-42d5-9b5c-4478b7b609b6-kube-api-access-pqkcs\") pod \"authorino-774f46b7b9-pd6zf\" (UID: \"6688a0d7-bdcc-42d5-9b5c-4478b7b609b6\") " pod="kuadrant-system/authorino-774f46b7b9-pd6zf"
Sep 11 13:31:13.475919 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.475877    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6688a0d7-bdcc-42d5-9b5c-4478b7b609b6-tls-cert\") pod \"authorino-774f46b7b9-pd6zf\" (UID: \"6688a0d7-bdcc-42d5-9b5c-4478b7b609b6\") " pod="kuadrant-system/authorino-774f46b7b9-pd6zf"
Sep 11 13:31:13.486226 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.486201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqkcs\" (UniqueName: \"kubernetes.io/projected/6688a0d7-bdcc-42d5-9b5c-4478b7b609b6-kube-api-access-pqkcs\") pod \"authorino-774f46b7b9-pd6zf\" (UID: \"6688a0d7-bdcc-42d5-9b5c-4478b7b609b6\") " pod="kuadrant-system/authorino-774f46b7b9-pd6zf"
Sep 11 13:31:13.648395 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.648309    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-774f46b7b9-pd6zf"
Sep 11 13:31:13.927050 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:13.927020    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-774f46b7b9-pd6zf"]
Sep 11 13:31:13.928065 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:31:13.928039    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6688a0d7_bdcc_42d5_9b5c_4478b7b609b6.slice/crio-92c33e37a4f66bddb810eb175cdb328a4cc7e20234ce539a9d6f36fda69c9936 WatchSource:0}: Error finding container 92c33e37a4f66bddb810eb175cdb328a4cc7e20234ce539a9d6f36fda69c9936: Status 404 returned error can't find the container with id 92c33e37a4f66bddb810eb175cdb328a4cc7e20234ce539a9d6f36fda69c9936
Sep 11 13:31:14.129579 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:14.129550    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-bpn65"
Sep 11 13:31:14.129579 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:14.129569    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-774f46b7b9-pd6zf" event={"ID":"6688a0d7-bdcc-42d5-9b5c-4478b7b609b6","Type":"ContainerStarted","Data":"92c33e37a4f66bddb810eb175cdb328a4cc7e20234ce539a9d6f36fda69c9936"}
Sep 11 13:31:14.137396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:14.137371    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-bpn65"
Sep 11 13:31:14.179223 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:14.179142    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pq7fd\" (UniqueName: \"kubernetes.io/projected/c990e6c6-2418-4bb8-ac6b-3b061681f87c-kube-api-access-pq7fd\") pod \"c990e6c6-2418-4bb8-ac6b-3b061681f87c\" (UID: \"c990e6c6-2418-4bb8-ac6b-3b061681f87c\") "
Sep 11 13:31:14.181305 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:14.181274    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c990e6c6-2418-4bb8-ac6b-3b061681f87c-kube-api-access-pq7fd" (OuterVolumeSpecName: "kube-api-access-pq7fd") pod "c990e6c6-2418-4bb8-ac6b-3b061681f87c" (UID: "c990e6c6-2418-4bb8-ac6b-3b061681f87c"). InnerVolumeSpecName "kube-api-access-pq7fd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:31:14.279709 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:14.279670    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq7fd\" (UniqueName: \"kubernetes.io/projected/c990e6c6-2418-4bb8-ac6b-3b061681f87c-kube-api-access-pq7fd\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:31:15.136287 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:15.136251    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-774f46b7b9-pd6zf" event={"ID":"6688a0d7-bdcc-42d5-9b5c-4478b7b609b6","Type":"ContainerStarted","Data":"2fb719283ef9761856f15280d976d7a5e33c38e908c67762505c35e43dcdccbe"}
Sep 11 13:31:15.136698 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:15.136397    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-bpn65"
Sep 11 13:31:15.155965 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:15.155916    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-774f46b7b9-pd6zf" podStartSLOduration=1.661805641 podStartE2EDuration="2.155887659s" totalImagesPullingTime="494.082018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:31:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:31:15.154440679 +0000 UTC m=+594.372978172" watchObservedRunningTime="2026-09-11 13:31:15.155887659 +0000 UTC m=+594.374425150"
Sep 11 13:31:15.186154 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:15.186116    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-bpn65"]
Sep 11 13:31:15.190944 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:15.190894    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-bpn65"]
Sep 11 13:31:15.316081 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:15.316044    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c990e6c6-2418-4bb8-ac6b-3b061681f87c" path="/var/lib/kubelet/pods/c990e6c6-2418-4bb8-ac6b-3b061681f87c/volumes"
Sep 11 13:31:21.270787 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:21.270752    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:31:21.271498 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:21.271473    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:31:57.800797 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:31:57.800748    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:32:21.349657 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:21.349466    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:32:29.150673 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:29.150634    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:32:36.751408 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:36.751367    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:32:40.156999 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.156960    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"]
Sep 11 13:32:40.160803 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.160782    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.164682 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.164662    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 11 13:32:40.164786 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.164697    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 13:32:40.164786 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.164709    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 13:32:40.164786 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.164668    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6z2kw\""
Sep 11 13:32:40.170843 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.170820    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"]
Sep 11 13:32:40.190612 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.190585    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b185795-5415-45d2-84e3-acef37f5dfe9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.190735 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.190616    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b185795-5415-45d2-84e3-acef37f5dfe9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.190735 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.190660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk9zf\" (UniqueName: \"kubernetes.io/projected/5b185795-5415-45d2-84e3-acef37f5dfe9-kube-api-access-pk9zf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.190735 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.190716    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b185795-5415-45d2-84e3-acef37f5dfe9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.190854 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.190747    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b185795-5415-45d2-84e3-acef37f5dfe9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.190854 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.190777    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b185795-5415-45d2-84e3-acef37f5dfe9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.291670 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.291635    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk9zf\" (UniqueName: \"kubernetes.io/projected/5b185795-5415-45d2-84e3-acef37f5dfe9-kube-api-access-pk9zf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.291841 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.291679    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b185795-5415-45d2-84e3-acef37f5dfe9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.291841 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.291711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b185795-5415-45d2-84e3-acef37f5dfe9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.291841 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.291743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b185795-5415-45d2-84e3-acef37f5dfe9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.291841 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.291825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b185795-5415-45d2-84e3-acef37f5dfe9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.292099 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.291846    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b185795-5415-45d2-84e3-acef37f5dfe9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.292099 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.292043    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b185795-5415-45d2-84e3-acef37f5dfe9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.292189 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.292124    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b185795-5415-45d2-84e3-acef37f5dfe9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.292189 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.292170    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b185795-5415-45d2-84e3-acef37f5dfe9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.294048 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.294030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b185795-5415-45d2-84e3-acef37f5dfe9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.294307 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.294290    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b185795-5415-45d2-84e3-acef37f5dfe9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.301255 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.301231    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk9zf\" (UniqueName: \"kubernetes.io/projected/5b185795-5415-45d2-84e3-acef37f5dfe9-kube-api-access-pk9zf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fkcct\" (UID: \"5b185795-5415-45d2-84e3-acef37f5dfe9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.472007 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.471926    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:40.748540 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.748511    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"]
Sep 11 13:32:40.749797 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:32:40.749756    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b185795_5415_45d2_84e3_acef37f5dfe9.slice/crio-20c096a0284084359d4cc5593a2c3a95b2f8f1bed0f865c5ab12ad7e6fd0cbe5 WatchSource:0}: Error finding container 20c096a0284084359d4cc5593a2c3a95b2f8f1bed0f865c5ab12ad7e6fd0cbe5: Status 404 returned error can't find the container with id 20c096a0284084359d4cc5593a2c3a95b2f8f1bed0f865c5ab12ad7e6fd0cbe5
Sep 11 13:32:40.751102 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:40.751083    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:32:41.469716 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:41.469684    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:32:41.581820 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:41.581787    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct" event={"ID":"5b185795-5415-45d2-84e3-acef37f5dfe9","Type":"ContainerStarted","Data":"20c096a0284084359d4cc5593a2c3a95b2f8f1bed0f865c5ab12ad7e6fd0cbe5"}
Sep 11 13:32:42.588981 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:42.588944    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct" event={"ID":"5b185795-5415-45d2-84e3-acef37f5dfe9","Type":"ContainerStarted","Data":"3d83c478fdb27bde5b9bcb1f0ec25b5bbe03ce0bfd8907b106328dea39eadf8d"}
Sep 11 13:32:42.612855 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:42.612804    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct" podStartSLOduration=1.567621068 podStartE2EDuration="2.612787565s" totalImagesPullingTime="1.045166497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:32:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:32:42.610696248 +0000 UTC m=+681.829233740" watchObservedRunningTime="2026-09-11 13:32:42.612787565 +0000 UTC m=+681.831325057"
Sep 11 13:32:50.472144 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:50.472109    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:50.472533 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:50.472168    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:50.484955 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:50.484926    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:50.640792 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:50.640760    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fkcct"
Sep 11 13:32:58.765124 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:32:58.765088    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:33:08.071254 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:08.071216    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:33:09.060117 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.060080    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"]
Sep 11 13:33:09.063989 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.063972    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.066790 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.066769    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 11 13:33:09.071266 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.071127    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"]
Sep 11 13:33:09.158248 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.158210    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.158248 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.158248    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.158674 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.158316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.158674 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.158357    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdzvw\" (UniqueName: \"kubernetes.io/projected/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-kube-api-access-qdzvw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.158674 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.158413    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.158674 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.158467    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.259125 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.259087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.259125 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.259129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.259324 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.259180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.259324 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.259205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdzvw\" (UniqueName: \"kubernetes.io/projected/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-kube-api-access-qdzvw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.259324 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.259277    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.259324 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.259318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.259655 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.259629    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.259737 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.259651    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.259737 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.259688    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.261562 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.261536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.261762 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.261748    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.269000 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.268979    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdzvw\" (UniqueName: \"kubernetes.io/projected/0df0572b-fff0-4b71-ac1e-bf358a8f43e4-kube-api-access-qdzvw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9\" (UID: \"0df0572b-fff0-4b71-ac1e-bf358a8f43e4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.374062 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.374031    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:09.648647 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.648619    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"]
Sep 11 13:33:09.649704 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:33:09.649669    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0df0572b_fff0_4b71_ac1e_bf358a8f43e4.slice/crio-9c77b77666f7376a465361a7031e789b28c2f6ec82e54cee1377881390de230e WatchSource:0}: Error finding container 9c77b77666f7376a465361a7031e789b28c2f6ec82e54cee1377881390de230e: Status 404 returned error can't find the container with id 9c77b77666f7376a465361a7031e789b28c2f6ec82e54cee1377881390de230e
Sep 11 13:33:09.727168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:09.727134    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9" event={"ID":"0df0572b-fff0-4b71-ac1e-bf358a8f43e4","Type":"ContainerStarted","Data":"9c77b77666f7376a465361a7031e789b28c2f6ec82e54cee1377881390de230e"}
Sep 11 13:33:10.734327 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:10.734295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9" event={"ID":"0df0572b-fff0-4b71-ac1e-bf358a8f43e4","Type":"ContainerStarted","Data":"f8f0461edb298333dde70e3398d35e2b66a3ba964eb7b74d080dec9ed597d2ff"}
Sep 11 13:33:10.756841 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:10.756789    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9" podStartSLOduration=1.578832281 podStartE2EDuration="1.756773658s" totalImagesPullingTime="177.941377ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:33:10.754233756 +0000 UTC m=+709.972771248" watchObservedRunningTime="2026-09-11 13:33:10.756773658 +0000 UTC m=+709.975311149"
Sep 11 13:33:15.855915 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:15.855823    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:33:19.375168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:19.375131    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:19.375586 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:19.375183    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:19.387663 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:19.387630    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:33:19.793288 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:33:19.793210    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9"
Sep 11 13:34:50.689664 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:50.689626    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-698d8fd7fb-r6z5s"]
Sep 11 13:34:50.692991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:50.692974    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-698d8fd7fb-r6z5s"
Sep 11 13:34:50.712513 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:50.712479    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-698d8fd7fb-r6z5s"]
Sep 11 13:34:50.848466 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:50.848409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/782d2109-57c6-4ec6-8ea4-3e6d26c19e1d-tls-cert\") pod \"authorino-698d8fd7fb-r6z5s\" (UID: \"782d2109-57c6-4ec6-8ea4-3e6d26c19e1d\") " pod="kuadrant-system/authorino-698d8fd7fb-r6z5s"
Sep 11 13:34:50.848466 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:50.848477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f99k2\" (UniqueName: \"kubernetes.io/projected/782d2109-57c6-4ec6-8ea4-3e6d26c19e1d-kube-api-access-f99k2\") pod \"authorino-698d8fd7fb-r6z5s\" (UID: \"782d2109-57c6-4ec6-8ea4-3e6d26c19e1d\") " pod="kuadrant-system/authorino-698d8fd7fb-r6z5s"
Sep 11 13:34:50.949703 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:50.949618    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/782d2109-57c6-4ec6-8ea4-3e6d26c19e1d-tls-cert\") pod \"authorino-698d8fd7fb-r6z5s\" (UID: \"782d2109-57c6-4ec6-8ea4-3e6d26c19e1d\") " pod="kuadrant-system/authorino-698d8fd7fb-r6z5s"
Sep 11 13:34:50.949703 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:50.949666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f99k2\" (UniqueName: \"kubernetes.io/projected/782d2109-57c6-4ec6-8ea4-3e6d26c19e1d-kube-api-access-f99k2\") pod \"authorino-698d8fd7fb-r6z5s\" (UID: \"782d2109-57c6-4ec6-8ea4-3e6d26c19e1d\") " pod="kuadrant-system/authorino-698d8fd7fb-r6z5s"
Sep 11 13:34:50.952127 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:50.952095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/782d2109-57c6-4ec6-8ea4-3e6d26c19e1d-tls-cert\") pod \"authorino-698d8fd7fb-r6z5s\" (UID: \"782d2109-57c6-4ec6-8ea4-3e6d26c19e1d\") " pod="kuadrant-system/authorino-698d8fd7fb-r6z5s"
Sep 11 13:34:50.958750 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:50.958718    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f99k2\" (UniqueName: \"kubernetes.io/projected/782d2109-57c6-4ec6-8ea4-3e6d26c19e1d-kube-api-access-f99k2\") pod \"authorino-698d8fd7fb-r6z5s\" (UID: \"782d2109-57c6-4ec6-8ea4-3e6d26c19e1d\") " pod="kuadrant-system/authorino-698d8fd7fb-r6z5s"
Sep 11 13:34:51.001587 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:51.001549    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-698d8fd7fb-r6z5s"
Sep 11 13:34:51.277218 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:51.277187    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-698d8fd7fb-r6z5s"]
Sep 11 13:34:51.278541 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:34:51.278515    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod782d2109_57c6_4ec6_8ea4_3e6d26c19e1d.slice/crio-c7204d54de85fda34570976f292c2cf0e4f709ff7edcdaa73c30a29812b2ee6a WatchSource:0}: Error finding container c7204d54de85fda34570976f292c2cf0e4f709ff7edcdaa73c30a29812b2ee6a: Status 404 returned error can't find the container with id c7204d54de85fda34570976f292c2cf0e4f709ff7edcdaa73c30a29812b2ee6a
Sep 11 13:34:52.277153 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:52.277049    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-698d8fd7fb-r6z5s" event={"ID":"782d2109-57c6-4ec6-8ea4-3e6d26c19e1d","Type":"ContainerStarted","Data":"3e68650f0c753d6bd2625cb68c40f81f422e839a699d41cd391128c1b078882d"}
Sep 11 13:34:52.277153 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:52.277114    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-698d8fd7fb-r6z5s" event={"ID":"782d2109-57c6-4ec6-8ea4-3e6d26c19e1d","Type":"ContainerStarted","Data":"c7204d54de85fda34570976f292c2cf0e4f709ff7edcdaa73c30a29812b2ee6a"}
Sep 11 13:34:52.305300 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:52.305241    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-698d8fd7fb-r6z5s" podStartSLOduration=1.601684821 podStartE2EDuration="2.305223001s" totalImagesPullingTime="703.53818ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:34:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:34:52.302517213 +0000 UTC m=+811.521054705" watchObservedRunningTime="2026-09-11 13:34:52.305223001 +0000 UTC m=+811.523760492"
Sep 11 13:34:52.360927 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:52.359923    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-774f46b7b9-pd6zf"]
Sep 11 13:34:52.360927 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:52.360131    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-774f46b7b9-pd6zf" podUID="6688a0d7-bdcc-42d5-9b5c-4478b7b609b6" containerName="authorino" containerID="cri-o://2fb719283ef9761856f15280d976d7a5e33c38e908c67762505c35e43dcdccbe" gracePeriod=30
Sep 11 13:34:52.619823 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:52.619794    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-774f46b7b9-pd6zf"
Sep 11 13:34:52.765311 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:52.765270    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6688a0d7-bdcc-42d5-9b5c-4478b7b609b6-tls-cert\") pod \"6688a0d7-bdcc-42d5-9b5c-4478b7b609b6\" (UID: \"6688a0d7-bdcc-42d5-9b5c-4478b7b609b6\") "
Sep 11 13:34:52.765507 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:52.765362    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqkcs\" (UniqueName: \"kubernetes.io/projected/6688a0d7-bdcc-42d5-9b5c-4478b7b609b6-kube-api-access-pqkcs\") pod \"6688a0d7-bdcc-42d5-9b5c-4478b7b609b6\" (UID: \"6688a0d7-bdcc-42d5-9b5c-4478b7b609b6\") "
Sep 11 13:34:52.767436 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:52.767401    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6688a0d7-bdcc-42d5-9b5c-4478b7b609b6-kube-api-access-pqkcs" (OuterVolumeSpecName: "kube-api-access-pqkcs") pod "6688a0d7-bdcc-42d5-9b5c-4478b7b609b6" (UID: "6688a0d7-bdcc-42d5-9b5c-4478b7b609b6"). InnerVolumeSpecName "kube-api-access-pqkcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:34:52.776556 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:52.776505    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6688a0d7-bdcc-42d5-9b5c-4478b7b609b6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "6688a0d7-bdcc-42d5-9b5c-4478b7b609b6" (UID: "6688a0d7-bdcc-42d5-9b5c-4478b7b609b6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:34:52.866109 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:52.866072    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6688a0d7-bdcc-42d5-9b5c-4478b7b609b6-tls-cert\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:34:52.866109 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:52.866105    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqkcs\" (UniqueName: \"kubernetes.io/projected/6688a0d7-bdcc-42d5-9b5c-4478b7b609b6-kube-api-access-pqkcs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:34:53.293634 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:53.283988    2580 generic.go:356] "Generic (PLEG): container finished" podID="6688a0d7-bdcc-42d5-9b5c-4478b7b609b6" containerID="2fb719283ef9761856f15280d976d7a5e33c38e908c67762505c35e43dcdccbe" exitCode=0
Sep 11 13:34:53.293634 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:53.284161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-774f46b7b9-pd6zf" event={"ID":"6688a0d7-bdcc-42d5-9b5c-4478b7b609b6","Type":"ContainerDied","Data":"2fb719283ef9761856f15280d976d7a5e33c38e908c67762505c35e43dcdccbe"}
Sep 11 13:34:53.293634 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:53.284214    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-774f46b7b9-pd6zf" event={"ID":"6688a0d7-bdcc-42d5-9b5c-4478b7b609b6","Type":"ContainerDied","Data":"92c33e37a4f66bddb810eb175cdb328a4cc7e20234ce539a9d6f36fda69c9936"}
Sep 11 13:34:53.293634 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:53.284239    2580 scope.go:117] "RemoveContainer" containerID="2fb719283ef9761856f15280d976d7a5e33c38e908c67762505c35e43dcdccbe"
Sep 11 13:34:53.293634 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:53.284430    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-774f46b7b9-pd6zf"
Sep 11 13:34:53.304803 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:53.304779    2580 scope.go:117] "RemoveContainer" containerID="2fb719283ef9761856f15280d976d7a5e33c38e908c67762505c35e43dcdccbe"
Sep 11 13:34:53.305165 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:34:53.305141    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fb719283ef9761856f15280d976d7a5e33c38e908c67762505c35e43dcdccbe\": container with ID starting with 2fb719283ef9761856f15280d976d7a5e33c38e908c67762505c35e43dcdccbe not found: ID does not exist" containerID="2fb719283ef9761856f15280d976d7a5e33c38e908c67762505c35e43dcdccbe"
Sep 11 13:34:53.305251 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:53.305172    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fb719283ef9761856f15280d976d7a5e33c38e908c67762505c35e43dcdccbe"} err="failed to get container status \"2fb719283ef9761856f15280d976d7a5e33c38e908c67762505c35e43dcdccbe\": rpc error: code = NotFound desc = could not find container \"2fb719283ef9761856f15280d976d7a5e33c38e908c67762505c35e43dcdccbe\": container with ID starting with 2fb719283ef9761856f15280d976d7a5e33c38e908c67762505c35e43dcdccbe not found: ID does not exist"
Sep 11 13:34:53.333817 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:53.333779    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-774f46b7b9-pd6zf"]
Sep 11 13:34:53.336499 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:53.336470    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-774f46b7b9-pd6zf"]
Sep 11 13:34:55.315394 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:34:55.315356    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6688a0d7-bdcc-42d5-9b5c-4478b7b609b6" path="/var/lib/kubelet/pods/6688a0d7-bdcc-42d5-9b5c-4478b7b609b6/volumes"
Sep 11 13:35:13.245495 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:13.245459    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7625-3a20a7"]
Sep 11 13:35:13.245844 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:13.245808    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6688a0d7-bdcc-42d5-9b5c-4478b7b609b6" containerName="authorino"
Sep 11 13:35:13.245844 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:13.245819    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6688a0d7-bdcc-42d5-9b5c-4478b7b609b6" containerName="authorino"
Sep 11 13:35:13.245956 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:13.245879    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6688a0d7-bdcc-42d5-9b5c-4478b7b609b6" containerName="authorino"
Sep 11 13:35:13.248994 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:13.248978    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7625-3a20a7"
Sep 11 13:35:13.252413 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:13.252387    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7ltrq\""
Sep 11 13:35:13.259670 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:13.259645    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7625-3a20a7"]
Sep 11 13:35:13.340674 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:13.340640    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq2nd\" (UniqueName: \"kubernetes.io/projected/26060af1-3693-4ad9-a9b0-f9ee154704d9-kube-api-access-tq2nd\") pod \"test-curl-7625-3a20a7\" (UID: \"26060af1-3693-4ad9-a9b0-f9ee154704d9\") " pod="opendatahub/test-curl-7625-3a20a7"
Sep 11 13:35:13.441334 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:13.441297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq2nd\" (UniqueName: \"kubernetes.io/projected/26060af1-3693-4ad9-a9b0-f9ee154704d9-kube-api-access-tq2nd\") pod \"test-curl-7625-3a20a7\" (UID: \"26060af1-3693-4ad9-a9b0-f9ee154704d9\") " pod="opendatahub/test-curl-7625-3a20a7"
Sep 11 13:35:13.456417 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:13.456382    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq2nd\" (UniqueName: \"kubernetes.io/projected/26060af1-3693-4ad9-a9b0-f9ee154704d9-kube-api-access-tq2nd\") pod \"test-curl-7625-3a20a7\" (UID: \"26060af1-3693-4ad9-a9b0-f9ee154704d9\") " pod="opendatahub/test-curl-7625-3a20a7"
Sep 11 13:35:13.558537 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:13.558450    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7625-3a20a7"
Sep 11 13:35:13.849436 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:13.849410    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7625-3a20a7"]
Sep 11 13:35:13.851591 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:35:13.851563    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26060af1_3693_4ad9_a9b0_f9ee154704d9.slice/crio-2450ef5788669681205cc2fa7e4744513309d5bc44ca54596368a0c0f3bdcee0 WatchSource:0}: Error finding container 2450ef5788669681205cc2fa7e4744513309d5bc44ca54596368a0c0f3bdcee0: Status 404 returned error can't find the container with id 2450ef5788669681205cc2fa7e4744513309d5bc44ca54596368a0c0f3bdcee0
Sep 11 13:35:14.399923 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:14.399867    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7625-3a20a7" event={"ID":"26060af1-3693-4ad9-a9b0-f9ee154704d9","Type":"ContainerStarted","Data":"2450ef5788669681205cc2fa7e4744513309d5bc44ca54596368a0c0f3bdcee0"}
Sep 11 13:35:16.193248 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:16.193210    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:35:17.419387 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:17.419286    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7625-3a20a7" event={"ID":"26060af1-3693-4ad9-a9b0-f9ee154704d9","Type":"ContainerStarted","Data":"4c74faa6f09cab3f86f18b70549542316ef52d85d660ecca1b8447f6eb1cd82b"}
Sep 11 13:35:17.448886 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:17.448834    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7625-3a20a7" podStartSLOduration=1.143980823 podStartE2EDuration="4.448815943s" totalImagesPullingTime="3.30483512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:17.448169038 +0000 UTC m=+836.666706531" watchObservedRunningTime="2026-09-11 13:35:17.448815943 +0000 UTC m=+836.667353480"
Sep 11 13:35:18.399550 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:18.399457    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7625-3a20a7"]
Sep 11 13:35:18.453611 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:18.453573    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7625-3a20a7"]
Sep 11 13:35:18.590326 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:18.590289    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7625-9d7d66"]
Sep 11 13:35:18.590641 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:18.590629    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26060af1-3693-4ad9-a9b0-f9ee154704d9" containerName="test-curl-7625-3a20a7"
Sep 11 13:35:18.590683 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:18.590643    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="26060af1-3693-4ad9-a9b0-f9ee154704d9" containerName="test-curl-7625-3a20a7"
Sep 11 13:35:18.590718 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:18.590707    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="26060af1-3693-4ad9-a9b0-f9ee154704d9" containerName="test-curl-7625-3a20a7"
Sep 11 13:35:18.594059 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:18.594044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7625-9d7d66"
Sep 11 13:35:18.664811 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:18.664730    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7625-9d7d66"]
Sep 11 13:35:18.696899 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:18.696865    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr8mn\" (UniqueName: \"kubernetes.io/projected/e118fc9e-66c6-4526-8c01-e1c3afc02911-kube-api-access-hr8mn\") pod \"test-curl-7625-9d7d66\" (UID: \"e118fc9e-66c6-4526-8c01-e1c3afc02911\") " pod="opendatahub/test-curl-7625-9d7d66"
Sep 11 13:35:18.798308 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:18.798266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr8mn\" (UniqueName: \"kubernetes.io/projected/e118fc9e-66c6-4526-8c01-e1c3afc02911-kube-api-access-hr8mn\") pod \"test-curl-7625-9d7d66\" (UID: \"e118fc9e-66c6-4526-8c01-e1c3afc02911\") " pod="opendatahub/test-curl-7625-9d7d66"
Sep 11 13:35:18.818003 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:18.817962    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr8mn\" (UniqueName: \"kubernetes.io/projected/e118fc9e-66c6-4526-8c01-e1c3afc02911-kube-api-access-hr8mn\") pod \"test-curl-7625-9d7d66\" (UID: \"e118fc9e-66c6-4526-8c01-e1c3afc02911\") " pod="opendatahub/test-curl-7625-9d7d66"
Sep 11 13:35:18.904753 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:18.904722    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7625-9d7d66"
Sep 11 13:35:19.194401 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:19.194374    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7625-9d7d66"]
Sep 11 13:35:19.196361 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:35:19.196327    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode118fc9e_66c6_4526_8c01_e1c3afc02911.slice/crio-51e55480f8c61edd54169167745814cdad4dfe1e23b1cec9728bdc18b31ad52f WatchSource:0}: Error finding container 51e55480f8c61edd54169167745814cdad4dfe1e23b1cec9728bdc18b31ad52f: Status 404 returned error can't find the container with id 51e55480f8c61edd54169167745814cdad4dfe1e23b1cec9728bdc18b31ad52f
Sep 11 13:35:19.433928 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:19.433863    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7625-9d7d66" event={"ID":"e118fc9e-66c6-4526-8c01-e1c3afc02911","Type":"ContainerStarted","Data":"633a6355208d6b68d1264601b288cb94498322c272a8b8e8e0cb6f8333bd8e7a"}
Sep 11 13:35:19.433928 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:19.433890    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7625-3a20a7" podUID="26060af1-3693-4ad9-a9b0-f9ee154704d9" containerName="test-curl-7625-3a20a7" containerID="cri-o://4c74faa6f09cab3f86f18b70549542316ef52d85d660ecca1b8447f6eb1cd82b" gracePeriod=2
Sep 11 13:35:19.433928 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:19.433930    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7625-9d7d66" event={"ID":"e118fc9e-66c6-4526-8c01-e1c3afc02911","Type":"ContainerStarted","Data":"51e55480f8c61edd54169167745814cdad4dfe1e23b1cec9728bdc18b31ad52f"}
Sep 11 13:35:19.462392 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:19.462297    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7625-9d7d66" podStartSLOduration=1.462282077 podStartE2EDuration="1.462282077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:19.459727566 +0000 UTC m=+838.678265051" watchObservedRunningTime="2026-09-11 13:35:19.462282077 +0000 UTC m=+838.680819569"
Sep 11 13:35:20.338962 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.338928    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7625-9d7d66"]
Sep 11 13:35:20.346324 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.346298    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7625-9d7d66"]
Sep 11 13:35:20.438923 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:35:20.438879    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7625-9d7d66" container="test-curl-7625-9d7d66"
Sep 11 13:35:20.576751 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.576716    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"]
Sep 11 13:35:20.577142 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.577099    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e118fc9e-66c6-4526-8c01-e1c3afc02911" containerName="test-curl-7625-9d7d66"
Sep 11 13:35:20.577142 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.577110    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e118fc9e-66c6-4526-8c01-e1c3afc02911" containerName="test-curl-7625-9d7d66"
Sep 11 13:35:20.577208 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.577169    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e118fc9e-66c6-4526-8c01-e1c3afc02911" containerName="test-curl-7625-9d7d66"
Sep 11 13:35:20.580187 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.580169    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"
Sep 11 13:35:20.583208 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.583178    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-082f62\""
Sep 11 13:35:20.583523 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.583505    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-082f62-dockercfg-c9mmc\""
Sep 11 13:35:20.592721 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.592670    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"]
Sep 11 13:35:20.596061 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.596044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"
Sep 11 13:35:20.603631 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.603609    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"]
Sep 11 13:35:20.609608 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.609582    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"]
Sep 11 13:35:20.713878 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.713834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h68ff\" (UniqueName: \"kubernetes.io/projected/6bfc7841-f1da-4e58-b222-90d47dc3eb34-kube-api-access-h68ff\") pod \"payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx\" (UID: \"6bfc7841-f1da-4e58-b222-90d47dc3eb34\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"
Sep 11 13:35:20.714095 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.713895    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2a76b58-aafd-4a9c-a999-4527253e8b9c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk\" (UID: \"f2a76b58-aafd-4a9c-a999-4527253e8b9c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"
Sep 11 13:35:20.714284 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.714084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bfc7841-f1da-4e58-b222-90d47dc3eb34-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx\" (UID: \"6bfc7841-f1da-4e58-b222-90d47dc3eb34\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"
Sep 11 13:35:20.714284 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.714121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99qhv\" (UniqueName: \"kubernetes.io/projected/f2a76b58-aafd-4a9c-a999-4527253e8b9c-kube-api-access-99qhv\") pod \"payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk\" (UID: \"f2a76b58-aafd-4a9c-a999-4527253e8b9c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"
Sep 11 13:35:20.815270 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.815230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bfc7841-f1da-4e58-b222-90d47dc3eb34-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx\" (UID: \"6bfc7841-f1da-4e58-b222-90d47dc3eb34\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"
Sep 11 13:35:20.815270 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.815270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99qhv\" (UniqueName: \"kubernetes.io/projected/f2a76b58-aafd-4a9c-a999-4527253e8b9c-kube-api-access-99qhv\") pod \"payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk\" (UID: \"f2a76b58-aafd-4a9c-a999-4527253e8b9c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"
Sep 11 13:35:20.815531 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.815302    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h68ff\" (UniqueName: \"kubernetes.io/projected/6bfc7841-f1da-4e58-b222-90d47dc3eb34-kube-api-access-h68ff\") pod \"payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx\" (UID: \"6bfc7841-f1da-4e58-b222-90d47dc3eb34\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"
Sep 11 13:35:20.815531 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.815419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2a76b58-aafd-4a9c-a999-4527253e8b9c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk\" (UID: \"f2a76b58-aafd-4a9c-a999-4527253e8b9c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"
Sep 11 13:35:20.816058 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.816035    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bfc7841-f1da-4e58-b222-90d47dc3eb34-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx\" (UID: \"6bfc7841-f1da-4e58-b222-90d47dc3eb34\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"
Sep 11 13:35:20.816156 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.816116    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2a76b58-aafd-4a9c-a999-4527253e8b9c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk\" (UID: \"f2a76b58-aafd-4a9c-a999-4527253e8b9c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"
Sep 11 13:35:20.834539 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.834509    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99qhv\" (UniqueName: \"kubernetes.io/projected/f2a76b58-aafd-4a9c-a999-4527253e8b9c-kube-api-access-99qhv\") pod \"payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk\" (UID: \"f2a76b58-aafd-4a9c-a999-4527253e8b9c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"
Sep 11 13:35:20.834934 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.834897    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h68ff\" (UniqueName: \"kubernetes.io/projected/6bfc7841-f1da-4e58-b222-90d47dc3eb34-kube-api-access-h68ff\") pod \"payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx\" (UID: \"6bfc7841-f1da-4e58-b222-90d47dc3eb34\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"
Sep 11 13:35:20.890115 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.890026    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"
Sep 11 13:35:20.907130 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:20.907101    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"
Sep 11 13:35:21.195441 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:35:21.195413    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a76b58_aafd_4a9c_a999_4527253e8b9c.slice/crio-4400d5ed82d9e3684f6798960b19598a185bfce21cea9092499ddbb640ba7e18 WatchSource:0}: Error finding container 4400d5ed82d9e3684f6798960b19598a185bfce21cea9092499ddbb640ba7e18: Status 404 returned error can't find the container with id 4400d5ed82d9e3684f6798960b19598a185bfce21cea9092499ddbb640ba7e18
Sep 11 13:35:21.196026 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:21.195954    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"]
Sep 11 13:35:21.239370 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:21.239347    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"]
Sep 11 13:35:21.241866 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:35:21.241836    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bfc7841_f1da_4e58_b222_90d47dc3eb34.slice/crio-0ec66a43a3f9875c9fee8394a001cc325d7cd138ea6da028a9c1f83fc62d28f7 WatchSource:0}: Error finding container 0ec66a43a3f9875c9fee8394a001cc325d7cd138ea6da028a9c1f83fc62d28f7: Status 404 returned error can't find the container with id 0ec66a43a3f9875c9fee8394a001cc325d7cd138ea6da028a9c1f83fc62d28f7
Sep 11 13:35:21.447113 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:21.447067    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk" event={"ID":"f2a76b58-aafd-4a9c-a999-4527253e8b9c","Type":"ContainerStarted","Data":"4400d5ed82d9e3684f6798960b19598a185bfce21cea9092499ddbb640ba7e18"}
Sep 11 13:35:21.448626 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:21.448594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx" event={"ID":"6bfc7841-f1da-4e58-b222-90d47dc3eb34","Type":"ContainerStarted","Data":"0ec66a43a3f9875c9fee8394a001cc325d7cd138ea6da028a9c1f83fc62d28f7"}
Sep 11 13:35:21.448737 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:21.448671    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7625-9d7d66" podUID="e118fc9e-66c6-4526-8c01-e1c3afc02911" containerName="test-curl-7625-9d7d66" containerID="cri-o://633a6355208d6b68d1264601b288cb94498322c272a8b8e8e0cb6f8333bd8e7a" gracePeriod=2
Sep 11 13:35:21.617188 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:21.617158    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7625-3a20a7"
Sep 11 13:35:21.724493 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:21.723784    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tq2nd\" (UniqueName: \"kubernetes.io/projected/26060af1-3693-4ad9-a9b0-f9ee154704d9-kube-api-access-tq2nd\") pod \"26060af1-3693-4ad9-a9b0-f9ee154704d9\" (UID: \"26060af1-3693-4ad9-a9b0-f9ee154704d9\") "
Sep 11 13:35:21.728591 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:21.728532    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26060af1-3693-4ad9-a9b0-f9ee154704d9-kube-api-access-tq2nd" (OuterVolumeSpecName: "kube-api-access-tq2nd") pod "26060af1-3693-4ad9-a9b0-f9ee154704d9" (UID: "26060af1-3693-4ad9-a9b0-f9ee154704d9"). InnerVolumeSpecName "kube-api-access-tq2nd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:35:21.825731 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:21.825693    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tq2nd\" (UniqueName: \"kubernetes.io/projected/26060af1-3693-4ad9-a9b0-f9ee154704d9-kube-api-access-tq2nd\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:35:22.459428 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:22.459386    2580 generic.go:356] "Generic (PLEG): container finished" podID="26060af1-3693-4ad9-a9b0-f9ee154704d9" containerID="4c74faa6f09cab3f86f18b70549542316ef52d85d660ecca1b8447f6eb1cd82b" exitCode=137
Sep 11 13:35:22.459620 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:22.459440    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7625-3a20a7"
Sep 11 13:35:22.459620 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:22.459488    2580 scope.go:117] "RemoveContainer" containerID="4c74faa6f09cab3f86f18b70549542316ef52d85d660ecca1b8447f6eb1cd82b"
Sep 11 13:35:22.474424 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:22.474193    2580 scope.go:117] "RemoveContainer" containerID="4c74faa6f09cab3f86f18b70549542316ef52d85d660ecca1b8447f6eb1cd82b"
Sep 11 13:35:22.474985 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:35:22.474849    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c74faa6f09cab3f86f18b70549542316ef52d85d660ecca1b8447f6eb1cd82b\": container with ID starting with 4c74faa6f09cab3f86f18b70549542316ef52d85d660ecca1b8447f6eb1cd82b not found: ID does not exist" containerID="4c74faa6f09cab3f86f18b70549542316ef52d85d660ecca1b8447f6eb1cd82b"
Sep 11 13:35:22.474985 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:22.474889    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c74faa6f09cab3f86f18b70549542316ef52d85d660ecca1b8447f6eb1cd82b"} err="failed to get container status \"4c74faa6f09cab3f86f18b70549542316ef52d85d660ecca1b8447f6eb1cd82b\": rpc error: code = NotFound desc = could not find container \"4c74faa6f09cab3f86f18b70549542316ef52d85d660ecca1b8447f6eb1cd82b\": container with ID starting with 4c74faa6f09cab3f86f18b70549542316ef52d85d660ecca1b8447f6eb1cd82b not found: ID does not exist"
Sep 11 13:35:22.484114 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:35:22.483757    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7625-3a20a7\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="26060af1-3693-4ad9-a9b0-f9ee154704d9" pod="opendatahub/test-curl-7625-3a20a7"
Sep 11 13:35:22.822830 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:22.822795    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"]
Sep 11 13:35:22.826164 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:22.826141    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"
Sep 11 13:35:22.833349 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:22.833330    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-371c38-dockercfg-7k5x9\""
Sep 11 13:35:22.834198 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:22.834174    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-371c38\""
Sep 11 13:35:22.836499 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:35:22.836470    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7625-3a20a7\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="26060af1-3693-4ad9-a9b0-f9ee154704d9" pod="opendatahub/test-curl-7625-3a20a7"
Sep 11 13:35:22.863252 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:22.863216    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"]
Sep 11 13:35:22.904089 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:22.904067    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"]
Sep 11 13:35:22.907986 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:22.907956    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"
Sep 11 13:35:22.911261 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:35:22.911225    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7625-3a20a7\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="26060af1-3693-4ad9-a9b0-f9ee154704d9" pod="opendatahub/test-curl-7625-3a20a7"
Sep 11 13:35:22.929831 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:22.929806    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"]
Sep 11 13:35:22.942633 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:22.942606    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfs6f\" (UniqueName: \"kubernetes.io/projected/4c4a728a-bb72-4b9f-b670-8b57bf22afdf-kube-api-access-vfs6f\") pod \"payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc\" (UID: \"4c4a728a-bb72-4b9f-b670-8b57bf22afdf\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"
Sep 11 13:35:22.942743 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:22.942652    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c4a728a-bb72-4b9f-b670-8b57bf22afdf-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc\" (UID: \"4c4a728a-bb72-4b9f-b670-8b57bf22afdf\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"
Sep 11 13:35:23.043204 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.043174    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0db7b13-a95c-4c5b-a021-80e0fe9eea4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s\" (UID: \"d0db7b13-a95c-4c5b-a021-80e0fe9eea4f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"
Sep 11 13:35:23.043332 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.043221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c4a728a-bb72-4b9f-b670-8b57bf22afdf-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc\" (UID: \"4c4a728a-bb72-4b9f-b670-8b57bf22afdf\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"
Sep 11 13:35:23.043392 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.043338    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfs6f\" (UniqueName: \"kubernetes.io/projected/4c4a728a-bb72-4b9f-b670-8b57bf22afdf-kube-api-access-vfs6f\") pod \"payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc\" (UID: \"4c4a728a-bb72-4b9f-b670-8b57bf22afdf\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"
Sep 11 13:35:23.043392 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.043364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tchr4\" (UniqueName: \"kubernetes.io/projected/d0db7b13-a95c-4c5b-a021-80e0fe9eea4f-kube-api-access-tchr4\") pod \"payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s\" (UID: \"d0db7b13-a95c-4c5b-a021-80e0fe9eea4f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"
Sep 11 13:35:23.044203 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.044177    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c4a728a-bb72-4b9f-b670-8b57bf22afdf-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc\" (UID: \"4c4a728a-bb72-4b9f-b670-8b57bf22afdf\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"
Sep 11 13:35:23.061445 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.061411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfs6f\" (UniqueName: \"kubernetes.io/projected/4c4a728a-bb72-4b9f-b670-8b57bf22afdf-kube-api-access-vfs6f\") pod \"payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc\" (UID: \"4c4a728a-bb72-4b9f-b670-8b57bf22afdf\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"
Sep 11 13:35:23.135847 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.135759    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"
Sep 11 13:35:23.144888 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.144419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tchr4\" (UniqueName: \"kubernetes.io/projected/d0db7b13-a95c-4c5b-a021-80e0fe9eea4f-kube-api-access-tchr4\") pod \"payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s\" (UID: \"d0db7b13-a95c-4c5b-a021-80e0fe9eea4f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"
Sep 11 13:35:23.144888 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.144474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0db7b13-a95c-4c5b-a021-80e0fe9eea4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s\" (UID: \"d0db7b13-a95c-4c5b-a021-80e0fe9eea4f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"
Sep 11 13:35:23.145219 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.145197    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0db7b13-a95c-4c5b-a021-80e0fe9eea4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s\" (UID: \"d0db7b13-a95c-4c5b-a021-80e0fe9eea4f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"
Sep 11 13:35:23.161216 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.161154    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tchr4\" (UniqueName: \"kubernetes.io/projected/d0db7b13-a95c-4c5b-a021-80e0fe9eea4f-kube-api-access-tchr4\") pod \"payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s\" (UID: \"d0db7b13-a95c-4c5b-a021-80e0fe9eea4f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"
Sep 11 13:35:23.313731 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.313695    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"
Sep 11 13:35:23.317330 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.317295    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26060af1-3693-4ad9-a9b0-f9ee154704d9" path="/var/lib/kubelet/pods/26060af1-3693-4ad9-a9b0-f9ee154704d9/volumes"
Sep 11 13:35:23.424127 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.424103    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"]
Sep 11 13:35:23.469512 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.469485    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc" event={"ID":"4c4a728a-bb72-4b9f-b670-8b57bf22afdf","Type":"ContainerStarted","Data":"1424f34cde78b450db325ed632879d82c1b63410478a59b7a325b16bec14462d"}
Sep 11 13:35:23.480758 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.480694    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk" event={"ID":"f2a76b58-aafd-4a9c-a999-4527253e8b9c","Type":"ContainerStarted","Data":"e89af5b31ff3f918dfac41f5343e8b1fa49dbbb677dace0241a0d0b4a13ca426"}
Sep 11 13:35:23.481087 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.480806    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"
Sep 11 13:35:23.483268 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.483238    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx" event={"ID":"6bfc7841-f1da-4e58-b222-90d47dc3eb34","Type":"ContainerStarted","Data":"e3e05254ae57ce540d1a3ab50f3db3b00e46b6fc2b7d13f3298dcac0a44d62d5"}
Sep 11 13:35:23.483407 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.483383    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"
Sep 11 13:35:23.503200 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.503144    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk" podStartSLOduration=1.800693519 podStartE2EDuration="3.503121997s" totalImagesPullingTime="1.702428478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:23.50158131 +0000 UTC m=+842.720118802" watchObservedRunningTime="2026-09-11 13:35:23.503121997 +0000 UTC m=+842.721659491"
Sep 11 13:35:23.528868 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.527875    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx" podStartSLOduration=1.88493694 podStartE2EDuration="3.527853627s" totalImagesPullingTime="1.642916687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:23.527568365 +0000 UTC m=+842.746105859" watchObservedRunningTime="2026-09-11 13:35:23.527853627 +0000 UTC m=+842.746391121"
Sep 11 13:35:23.602636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.602590    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"]
Sep 11 13:35:23.607558 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:35:23.607506    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0db7b13_a95c_4c5b_a021_80e0fe9eea4f.slice/crio-0ddc8ab6371921ea65566d417a9764500866635587c908f3742363eebfa8f113 WatchSource:0}: Error finding container 0ddc8ab6371921ea65566d417a9764500866635587c908f3742363eebfa8f113: Status 404 returned error can't find the container with id 0ddc8ab6371921ea65566d417a9764500866635587c908f3742363eebfa8f113
Sep 11 13:35:23.650577 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.650556    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7625-9d7d66"
Sep 11 13:35:23.750440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.750346    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hr8mn\" (UniqueName: \"kubernetes.io/projected/e118fc9e-66c6-4526-8c01-e1c3afc02911-kube-api-access-hr8mn\") pod \"e118fc9e-66c6-4526-8c01-e1c3afc02911\" (UID: \"e118fc9e-66c6-4526-8c01-e1c3afc02911\") "
Sep 11 13:35:23.752340 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.752311    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e118fc9e-66c6-4526-8c01-e1c3afc02911-kube-api-access-hr8mn" (OuterVolumeSpecName: "kube-api-access-hr8mn") pod "e118fc9e-66c6-4526-8c01-e1c3afc02911" (UID: "e118fc9e-66c6-4526-8c01-e1c3afc02911"). InnerVolumeSpecName "kube-api-access-hr8mn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:35:23.851203 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:23.851166    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr8mn\" (UniqueName: \"kubernetes.io/projected/e118fc9e-66c6-4526-8c01-e1c3afc02911-kube-api-access-hr8mn\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:35:24.490722 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:24.490691    2580 generic.go:356] "Generic (PLEG): container finished" podID="e118fc9e-66c6-4526-8c01-e1c3afc02911" containerID="633a6355208d6b68d1264601b288cb94498322c272a8b8e8e0cb6f8333bd8e7a" exitCode=137
Sep 11 13:35:24.490942 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:24.490740    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7625-9d7d66"
Sep 11 13:35:24.490942 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:24.490785    2580 scope.go:117] "RemoveContainer" containerID="633a6355208d6b68d1264601b288cb94498322c272a8b8e8e0cb6f8333bd8e7a"
Sep 11 13:35:24.492789 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:24.492679    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s" event={"ID":"d0db7b13-a95c-4c5b-a021-80e0fe9eea4f","Type":"ContainerStarted","Data":"0f179142949109fe5ebf5c9a41ddf336ffcc69d43d221bb5cd4c747a0dadc60a"}
Sep 11 13:35:24.492789 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:24.492729    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s" event={"ID":"d0db7b13-a95c-4c5b-a021-80e0fe9eea4f","Type":"ContainerStarted","Data":"0ddc8ab6371921ea65566d417a9764500866635587c908f3742363eebfa8f113"}
Sep 11 13:35:24.492993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:24.492851    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"
Sep 11 13:35:24.494459 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:24.494419    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc" event={"ID":"4c4a728a-bb72-4b9f-b670-8b57bf22afdf","Type":"ContainerStarted","Data":"bcfc4813b63c3d3485fb85ff0897d5348fdf1b82767b899718f32b25d671caed"}
Sep 11 13:35:24.494580 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:24.494476    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"
Sep 11 13:35:24.502072 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:24.501845    2580 scope.go:117] "RemoveContainer" containerID="633a6355208d6b68d1264601b288cb94498322c272a8b8e8e0cb6f8333bd8e7a"
Sep 11 13:35:24.502142 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:35:24.502125    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"633a6355208d6b68d1264601b288cb94498322c272a8b8e8e0cb6f8333bd8e7a\": container with ID starting with 633a6355208d6b68d1264601b288cb94498322c272a8b8e8e0cb6f8333bd8e7a not found: ID does not exist" containerID="633a6355208d6b68d1264601b288cb94498322c272a8b8e8e0cb6f8333bd8e7a"
Sep 11 13:35:24.502182 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:24.502150    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"633a6355208d6b68d1264601b288cb94498322c272a8b8e8e0cb6f8333bd8e7a"} err="failed to get container status \"633a6355208d6b68d1264601b288cb94498322c272a8b8e8e0cb6f8333bd8e7a\": rpc error: code = NotFound desc = could not find container \"633a6355208d6b68d1264601b288cb94498322c272a8b8e8e0cb6f8333bd8e7a\": container with ID starting with 633a6355208d6b68d1264601b288cb94498322c272a8b8e8e0cb6f8333bd8e7a not found: ID does not exist"
Sep 11 13:35:24.516388 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:24.516349    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s" podStartSLOduration=2.516336269 podStartE2EDuration="2.516336269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:24.515658469 +0000 UTC m=+843.734195965" watchObservedRunningTime="2026-09-11 13:35:24.516336269 +0000 UTC m=+843.734873760"
Sep 11 13:35:24.518285 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:35:24.518264    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7625-9d7d66\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="e118fc9e-66c6-4526-8c01-e1c3afc02911" pod="opendatahub/test-curl-7625-9d7d66"
Sep 11 13:35:24.538107 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:24.538067    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc" podStartSLOduration=2.538054763 podStartE2EDuration="2.538054763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:24.536750335 +0000 UTC m=+843.755287827" watchObservedRunningTime="2026-09-11 13:35:24.538054763 +0000 UTC m=+843.756592256"
Sep 11 13:35:25.316672 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:25.316636    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e118fc9e-66c6-4526-8c01-e1c3afc02911" path="/var/lib/kubelet/pods/e118fc9e-66c6-4526-8c01-e1c3afc02911/volumes"
Sep 11 13:35:27.176519 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:27.176486    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:35:28.144937 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.144888    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"]
Sep 11 13:35:28.153440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.153395    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"
Sep 11 13:35:28.156815 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.156783    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-f9f26e-dockercfg-vwnm4\""
Sep 11 13:35:28.157000 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.156822    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-f9f26e\""
Sep 11 13:35:28.166111 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.166081    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"]
Sep 11 13:35:28.289227 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.289182    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt2xn\" (UniqueName: \"kubernetes.io/projected/36b3926c-a4e9-4aea-904c-67ab2e684ecd-kube-api-access-kt2xn\") pod \"payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc\" (UID: \"36b3926c-a4e9-4aea-904c-67ab2e684ecd\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"
Sep 11 13:35:28.289227 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.289235    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b3926c-a4e9-4aea-904c-67ab2e684ecd-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc\" (UID: \"36b3926c-a4e9-4aea-904c-67ab2e684ecd\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"
Sep 11 13:35:28.364897 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.364850    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"]
Sep 11 13:35:28.368467 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.368451    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"
Sep 11 13:35:28.384660 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.384630    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"]
Sep 11 13:35:28.390577 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.390548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b3926c-a4e9-4aea-904c-67ab2e684ecd-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc\" (UID: \"36b3926c-a4e9-4aea-904c-67ab2e684ecd\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"
Sep 11 13:35:28.390708 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.390681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt2xn\" (UniqueName: \"kubernetes.io/projected/36b3926c-a4e9-4aea-904c-67ab2e684ecd-kube-api-access-kt2xn\") pod \"payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc\" (UID: \"36b3926c-a4e9-4aea-904c-67ab2e684ecd\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"
Sep 11 13:35:28.391313 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.391293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b3926c-a4e9-4aea-904c-67ab2e684ecd-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc\" (UID: \"36b3926c-a4e9-4aea-904c-67ab2e684ecd\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"
Sep 11 13:35:28.410820 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.410759    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt2xn\" (UniqueName: \"kubernetes.io/projected/36b3926c-a4e9-4aea-904c-67ab2e684ecd-kube-api-access-kt2xn\") pod \"payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc\" (UID: \"36b3926c-a4e9-4aea-904c-67ab2e684ecd\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"
Sep 11 13:35:28.465851 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.465820    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"
Sep 11 13:35:28.491448 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.491396    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2aa8689d-f67f-4f5e-a59a-8ce70236b9ab-plugins-config-volume\") pod \"payload-processing-e2e-autores-f9f26e-666c574685-kr5d9\" (UID: \"2aa8689d-f67f-4f5e-a59a-8ce70236b9ab\") " pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"
Sep 11 13:35:28.491612 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.491456    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krsjm\" (UniqueName: \"kubernetes.io/projected/2aa8689d-f67f-4f5e-a59a-8ce70236b9ab-kube-api-access-krsjm\") pod \"payload-processing-e2e-autores-f9f26e-666c574685-kr5d9\" (UID: \"2aa8689d-f67f-4f5e-a59a-8ce70236b9ab\") " pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"
Sep 11 13:35:28.591982 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.591939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2aa8689d-f67f-4f5e-a59a-8ce70236b9ab-plugins-config-volume\") pod \"payload-processing-e2e-autores-f9f26e-666c574685-kr5d9\" (UID: \"2aa8689d-f67f-4f5e-a59a-8ce70236b9ab\") " pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"
Sep 11 13:35:28.592168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.591992    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krsjm\" (UniqueName: \"kubernetes.io/projected/2aa8689d-f67f-4f5e-a59a-8ce70236b9ab-kube-api-access-krsjm\") pod \"payload-processing-e2e-autores-f9f26e-666c574685-kr5d9\" (UID: \"2aa8689d-f67f-4f5e-a59a-8ce70236b9ab\") " pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"
Sep 11 13:35:28.592643 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.592621    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2aa8689d-f67f-4f5e-a59a-8ce70236b9ab-plugins-config-volume\") pod \"payload-processing-e2e-autores-f9f26e-666c574685-kr5d9\" (UID: \"2aa8689d-f67f-4f5e-a59a-8ce70236b9ab\") " pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"
Sep 11 13:35:28.605233 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.605204    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krsjm\" (UniqueName: \"kubernetes.io/projected/2aa8689d-f67f-4f5e-a59a-8ce70236b9ab-kube-api-access-krsjm\") pod \"payload-processing-e2e-autores-f9f26e-666c574685-kr5d9\" (UID: \"2aa8689d-f67f-4f5e-a59a-8ce70236b9ab\") " pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"
Sep 11 13:35:28.678984 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.678867    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"
Sep 11 13:35:28.831171 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.831144    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"]
Sep 11 13:35:28.963140 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:28.963115    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"]
Sep 11 13:35:28.963780 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:35:28.963759    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aa8689d_f67f_4f5e_a59a_8ce70236b9ab.slice/crio-b15664947e2a57587afd27abed2c3544bb0fe98ac19be19eca34e75443cdf9a3 WatchSource:0}: Error finding container b15664947e2a57587afd27abed2c3544bb0fe98ac19be19eca34e75443cdf9a3: Status 404 returned error can't find the container with id b15664947e2a57587afd27abed2c3544bb0fe98ac19be19eca34e75443cdf9a3
Sep 11 13:35:29.532395 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:29.532359    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9" event={"ID":"2aa8689d-f67f-4f5e-a59a-8ce70236b9ab","Type":"ContainerStarted","Data":"90b582c17abd2dac3c6b6da29721b7881cdf064e6f5208d98e376fec505c0466"}
Sep 11 13:35:29.532785 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:29.532410    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9" event={"ID":"2aa8689d-f67f-4f5e-a59a-8ce70236b9ab","Type":"ContainerStarted","Data":"b15664947e2a57587afd27abed2c3544bb0fe98ac19be19eca34e75443cdf9a3"}
Sep 11 13:35:29.532785 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:29.532460    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"
Sep 11 13:35:29.534171 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:29.534143    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc" event={"ID":"36b3926c-a4e9-4aea-904c-67ab2e684ecd","Type":"ContainerStarted","Data":"9f0c5bd46c08aa282f359ad667011fa428acc79a9fb45724a3c102d40d2ca7c8"}
Sep 11 13:35:29.534285 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:29.534184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc" event={"ID":"36b3926c-a4e9-4aea-904c-67ab2e684ecd","Type":"ContainerStarted","Data":"f6c47fb1c3dc8a7f9f5d2e55c11dcbe3add4e9f603b03d8280a6b24378654dec"}
Sep 11 13:35:29.534285 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:29.534270    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"
Sep 11 13:35:29.557481 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:29.557431    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9" podStartSLOduration=1.5574154930000002 podStartE2EDuration="1.557415493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:29.555138132 +0000 UTC m=+848.773675628" watchObservedRunningTime="2026-09-11 13:35:29.557415493 +0000 UTC m=+848.775952986"
Sep 11 13:35:29.578308 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:29.578258    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc" podStartSLOduration=1.57824421 podStartE2EDuration="1.57824421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:29.576675035 +0000 UTC m=+848.795212518" watchObservedRunningTime="2026-09-11 13:35:29.57824421 +0000 UTC m=+848.796781696"
Sep 11 13:35:34.107988 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:34.107952    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"]
Sep 11 13:35:34.111595 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:34.111577    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"
Sep 11 13:35:34.115752 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:34.115728    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ae1d6b61-dockercfg-vddpj\""
Sep 11 13:35:34.116071 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:34.116052    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ae1d6b61\""
Sep 11 13:35:34.132225 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:34.132202    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"]
Sep 11 13:35:34.244809 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:34.244777    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a16abcc-2abc-42a4-a687-e79bb7405e49-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8\" (UID: \"2a16abcc-2abc-42a4-a687-e79bb7405e49\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"
Sep 11 13:35:34.244978 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:34.244846    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hqbn\" (UniqueName: \"kubernetes.io/projected/2a16abcc-2abc-42a4-a687-e79bb7405e49-kube-api-access-7hqbn\") pod \"payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8\" (UID: \"2a16abcc-2abc-42a4-a687-e79bb7405e49\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"
Sep 11 13:35:34.346273 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:34.346231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a16abcc-2abc-42a4-a687-e79bb7405e49-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8\" (UID: \"2a16abcc-2abc-42a4-a687-e79bb7405e49\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"
Sep 11 13:35:34.346426 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:34.346327    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hqbn\" (UniqueName: \"kubernetes.io/projected/2a16abcc-2abc-42a4-a687-e79bb7405e49-kube-api-access-7hqbn\") pod \"payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8\" (UID: \"2a16abcc-2abc-42a4-a687-e79bb7405e49\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"
Sep 11 13:35:34.346978 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:34.346959    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a16abcc-2abc-42a4-a687-e79bb7405e49-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8\" (UID: \"2a16abcc-2abc-42a4-a687-e79bb7405e49\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"
Sep 11 13:35:34.362278 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:34.362211    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hqbn\" (UniqueName: \"kubernetes.io/projected/2a16abcc-2abc-42a4-a687-e79bb7405e49-kube-api-access-7hqbn\") pod \"payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8\" (UID: \"2a16abcc-2abc-42a4-a687-e79bb7405e49\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"
Sep 11 13:35:34.422415 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:34.422383    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"
Sep 11 13:35:34.497053 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:34.497020    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"
Sep 11 13:35:34.497253 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:34.497238    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"
Sep 11 13:35:34.722600 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:35:34.722572    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a16abcc_2abc_42a4_a687_e79bb7405e49.slice/crio-af1c7cee2ee65c8b6925785569e254765e6b12d1d9b6583d4d7cf27a5a81b513 WatchSource:0}: Error finding container af1c7cee2ee65c8b6925785569e254765e6b12d1d9b6583d4d7cf27a5a81b513: Status 404 returned error can't find the container with id af1c7cee2ee65c8b6925785569e254765e6b12d1d9b6583d4d7cf27a5a81b513
Sep 11 13:35:34.723137 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:34.723115    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"]
Sep 11 13:35:35.503347 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:35.503309    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"
Sep 11 13:35:35.503745 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:35.503405    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"
Sep 11 13:35:35.568155 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:35.568116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8" event={"ID":"2a16abcc-2abc-42a4-a687-e79bb7405e49","Type":"ContainerStarted","Data":"3238b5bea2715766e26c47c7589ec4fb6df9c76d123658fcd849ea43542bbae9"}
Sep 11 13:35:35.568326 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:35.568196    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8" event={"ID":"2a16abcc-2abc-42a4-a687-e79bb7405e49","Type":"ContainerStarted","Data":"af1c7cee2ee65c8b6925785569e254765e6b12d1d9b6583d4d7cf27a5a81b513"}
Sep 11 13:35:35.568326 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:35.568226    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"
Sep 11 13:35:35.625684 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:35.625626    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8" podStartSLOduration=1.625609903 podStartE2EDuration="1.625609903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:35.625076666 +0000 UTC m=+854.843614159" watchObservedRunningTime="2026-09-11 13:35:35.625609903 +0000 UTC m=+854.844147395"
Sep 11 13:35:40.541306 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:40.541264    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"
Sep 11 13:35:40.541688 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:40.541343    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"
Sep 11 13:35:46.575250 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:46.575220    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"
Sep 11 13:35:48.076060 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:48.076023    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"]
Sep 11 13:35:48.084668 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:48.084638    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"
Sep 11 13:35:48.088306 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:48.088277    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-a649d3-dockercfg-sdp64\""
Sep 11 13:35:48.088306 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:48.088297    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-a649d3\""
Sep 11 13:35:48.097260 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:48.093688    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"]
Sep 11 13:35:48.174960 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:48.174927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqb5q\" (UniqueName: \"kubernetes.io/projected/64567a04-9955-485b-a593-77e9733d51c7-kube-api-access-jqb5q\") pod \"payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97\" (UID: \"64567a04-9955-485b-a593-77e9733d51c7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"
Sep 11 13:35:48.175114 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:48.174983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64567a04-9955-485b-a593-77e9733d51c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97\" (UID: \"64567a04-9955-485b-a593-77e9733d51c7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"
Sep 11 13:35:48.275953 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:48.275891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqb5q\" (UniqueName: \"kubernetes.io/projected/64567a04-9955-485b-a593-77e9733d51c7-kube-api-access-jqb5q\") pod \"payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97\" (UID: \"64567a04-9955-485b-a593-77e9733d51c7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"
Sep 11 13:35:48.276124 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:48.275985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64567a04-9955-485b-a593-77e9733d51c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97\" (UID: \"64567a04-9955-485b-a593-77e9733d51c7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"
Sep 11 13:35:48.276717 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:48.276697    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64567a04-9955-485b-a593-77e9733d51c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97\" (UID: \"64567a04-9955-485b-a593-77e9733d51c7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"
Sep 11 13:35:48.284315 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:48.284291    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqb5q\" (UniqueName: \"kubernetes.io/projected/64567a04-9955-485b-a593-77e9733d51c7-kube-api-access-jqb5q\") pod \"payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97\" (UID: \"64567a04-9955-485b-a593-77e9733d51c7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"
Sep 11 13:35:48.403043 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:48.402951    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"
Sep 11 13:35:48.678596 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:48.678567    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"]
Sep 11 13:35:48.679826 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:35:48.679797    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64567a04_9955_485b_a593_77e9733d51c7.slice/crio-7b8efca744c67ed3cbfc420585bce2fcf6fe227863f46af21e0646d0975830a2 WatchSource:0}: Error finding container 7b8efca744c67ed3cbfc420585bce2fcf6fe227863f46af21e0646d0975830a2: Status 404 returned error can't find the container with id 7b8efca744c67ed3cbfc420585bce2fcf6fe227863f46af21e0646d0975830a2
Sep 11 13:35:49.255421 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:49.255387    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn"]
Sep 11 13:35:49.259389 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:49.259372    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn"
Sep 11 13:35:49.262882 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:49.262862    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:35:49.262995 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:49.262928    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:35:49.269808 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:49.269781    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn"]
Sep 11 13:35:49.286349 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:49.286324    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c1ad9e-57da-4d5f-9a2a-f7fdd6697173-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn\" (UID: \"64c1ad9e-57da-4d5f-9a2a-f7fdd6697173\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn"
Sep 11 13:35:49.387265 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:49.387232    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c1ad9e-57da-4d5f-9a2a-f7fdd6697173-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn\" (UID: \"64c1ad9e-57da-4d5f-9a2a-f7fdd6697173\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn"
Sep 11 13:35:49.387940 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:49.387893    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c1ad9e-57da-4d5f-9a2a-f7fdd6697173-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn\" (UID: \"64c1ad9e-57da-4d5f-9a2a-f7fdd6697173\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn"
Sep 11 13:35:49.570644 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:49.570613    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn"
Sep 11 13:35:49.656562 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:49.656526    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97" event={"ID":"64567a04-9955-485b-a593-77e9733d51c7","Type":"ContainerStarted","Data":"c9a6522042cfea12e7edea4c85aaec08c0081562a4eddd2a384faa1c4d0dd905"}
Sep 11 13:35:49.656706 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:49.656585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97" event={"ID":"64567a04-9955-485b-a593-77e9733d51c7","Type":"ContainerStarted","Data":"7b8efca744c67ed3cbfc420585bce2fcf6fe227863f46af21e0646d0975830a2"}
Sep 11 13:35:49.656706 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:49.656614    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"
Sep 11 13:35:49.690778 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:49.690731    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97" podStartSLOduration=1.69071595 podStartE2EDuration="1.69071595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:49.689774727 +0000 UTC m=+868.908312220" watchObservedRunningTime="2026-09-11 13:35:49.69071595 +0000 UTC m=+868.909253489"
Sep 11 13:35:49.848879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:49.848852    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn"]
Sep 11 13:35:49.849946 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:35:49.849920    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64c1ad9e_57da_4d5f_9a2a_f7fdd6697173.slice/crio-60b22ed93896d58fd3c1114e1260237c8979dbbba1b2bd1c8bdcd9e0d49253f8 WatchSource:0}: Error finding container 60b22ed93896d58fd3c1114e1260237c8979dbbba1b2bd1c8bdcd9e0d49253f8: Status 404 returned error can't find the container with id 60b22ed93896d58fd3c1114e1260237c8979dbbba1b2bd1c8bdcd9e0d49253f8
Sep 11 13:35:50.665542 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:50.665255    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn" event={"ID":"64c1ad9e-57da-4d5f-9a2a-f7fdd6697173","Type":"ContainerStarted","Data":"60b22ed93896d58fd3c1114e1260237c8979dbbba1b2bd1c8bdcd9e0d49253f8"}
Sep 11 13:35:52.677397 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:52.677363    2580 generic.go:356] "Generic (PLEG): container finished" podID="64c1ad9e-57da-4d5f-9a2a-f7fdd6697173" containerID="bafb5958bc5e5f8b708fd15e20876464056bf80aa99fe6a4b5a750b39347a4c5" exitCode=0
Sep 11 13:35:52.677854 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:52.677448    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn" event={"ID":"64c1ad9e-57da-4d5f-9a2a-f7fdd6697173","Type":"ContainerDied","Data":"bafb5958bc5e5f8b708fd15e20876464056bf80aa99fe6a4b5a750b39347a4c5"}
Sep 11 13:35:53.818770 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:53.818747    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn"
Sep 11 13:35:53.930816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:53.930780    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c1ad9e-57da-4d5f-9a2a-f7fdd6697173-maas-api-service-ca\") pod \"64c1ad9e-57da-4d5f-9a2a-f7fdd6697173\" (UID: \"64c1ad9e-57da-4d5f-9a2a-f7fdd6697173\") "
Sep 11 13:35:53.931158 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:53.931135    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64c1ad9e-57da-4d5f-9a2a-f7fdd6697173-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "64c1ad9e-57da-4d5f-9a2a-f7fdd6697173" (UID: "64c1ad9e-57da-4d5f-9a2a-f7fdd6697173"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:35:54.031768 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:54.031675    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c1ad9e-57da-4d5f-9a2a-f7fdd6697173-maas-api-service-ca\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:35:54.693951 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:54.693915    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn"
Sep 11 13:35:54.693951 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:54.693935    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn" event={"ID":"64c1ad9e-57da-4d5f-9a2a-f7fdd6697173","Type":"ContainerDied","Data":"60b22ed93896d58fd3c1114e1260237c8979dbbba1b2bd1c8bdcd9e0d49253f8"}
Sep 11 13:35:54.694161 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:54.693985    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60b22ed93896d58fd3c1114e1260237c8979dbbba1b2bd1c8bdcd9e0d49253f8"
Sep 11 13:35:54.925810 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:35:54.925774    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:36:00.668007 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:00.667974    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"
Sep 11 13:36:03.918158 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:03.918116    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"]
Sep 11 13:36:03.918667 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:03.918366    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8" podUID="2a16abcc-2abc-42a4-a687-e79bb7405e49" containerName="payload-pre-processing" containerID="cri-o://3238b5bea2715766e26c47c7589ec4fb6df9c76d123658fcd849ea43542bbae9" gracePeriod=30
Sep 11 13:36:04.063524 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.063491    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn"]
Sep 11 13:36:04.068433 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.068396    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ae1d6b61-cctsn"]
Sep 11 13:36:04.179344 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.179256    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"
Sep 11 13:36:04.222779 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.222749    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hqbn\" (UniqueName: \"kubernetes.io/projected/2a16abcc-2abc-42a4-a687-e79bb7405e49-kube-api-access-7hqbn\") pod \"2a16abcc-2abc-42a4-a687-e79bb7405e49\" (UID: \"2a16abcc-2abc-42a4-a687-e79bb7405e49\") "
Sep 11 13:36:04.222987 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.222862    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a16abcc-2abc-42a4-a687-e79bb7405e49-plugins-config-volume\") pod \"2a16abcc-2abc-42a4-a687-e79bb7405e49\" (UID: \"2a16abcc-2abc-42a4-a687-e79bb7405e49\") "
Sep 11 13:36:04.223288 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.223262    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a16abcc-2abc-42a4-a687-e79bb7405e49-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2a16abcc-2abc-42a4-a687-e79bb7405e49" (UID: "2a16abcc-2abc-42a4-a687-e79bb7405e49"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:36:04.225164 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.225128    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a16abcc-2abc-42a4-a687-e79bb7405e49-kube-api-access-7hqbn" (OuterVolumeSpecName: "kube-api-access-7hqbn") pod "2a16abcc-2abc-42a4-a687-e79bb7405e49" (UID: "2a16abcc-2abc-42a4-a687-e79bb7405e49"). InnerVolumeSpecName "kube-api-access-7hqbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:36:04.323973 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.323939    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a16abcc-2abc-42a4-a687-e79bb7405e49-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:36:04.323973 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.323968    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hqbn\" (UniqueName: \"kubernetes.io/projected/2a16abcc-2abc-42a4-a687-e79bb7405e49-kube-api-access-7hqbn\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:36:04.752288 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.752252    2580 generic.go:356] "Generic (PLEG): container finished" podID="2a16abcc-2abc-42a4-a687-e79bb7405e49" containerID="3238b5bea2715766e26c47c7589ec4fb6df9c76d123658fcd849ea43542bbae9" exitCode=0
Sep 11 13:36:04.752494 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.752294    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"
Sep 11 13:36:04.752494 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.752334    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8" event={"ID":"2a16abcc-2abc-42a4-a687-e79bb7405e49","Type":"ContainerDied","Data":"3238b5bea2715766e26c47c7589ec4fb6df9c76d123658fcd849ea43542bbae9"}
Sep 11 13:36:04.752494 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.752397    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8" event={"ID":"2a16abcc-2abc-42a4-a687-e79bb7405e49","Type":"ContainerDied","Data":"af1c7cee2ee65c8b6925785569e254765e6b12d1d9b6583d4d7cf27a5a81b513"}
Sep 11 13:36:04.752494 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.752426    2580 scope.go:117] "RemoveContainer" containerID="3238b5bea2715766e26c47c7589ec4fb6df9c76d123658fcd849ea43542bbae9"
Sep 11 13:36:04.761921 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.761892    2580 scope.go:117] "RemoveContainer" containerID="3238b5bea2715766e26c47c7589ec4fb6df9c76d123658fcd849ea43542bbae9"
Sep 11 13:36:04.762188 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:36:04.762170    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3238b5bea2715766e26c47c7589ec4fb6df9c76d123658fcd849ea43542bbae9\": container with ID starting with 3238b5bea2715766e26c47c7589ec4fb6df9c76d123658fcd849ea43542bbae9 not found: ID does not exist" containerID="3238b5bea2715766e26c47c7589ec4fb6df9c76d123658fcd849ea43542bbae9"
Sep 11 13:36:04.762240 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.762195    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3238b5bea2715766e26c47c7589ec4fb6df9c76d123658fcd849ea43542bbae9"} err="failed to get container status \"3238b5bea2715766e26c47c7589ec4fb6df9c76d123658fcd849ea43542bbae9\": rpc error: code = NotFound desc = could not find container \"3238b5bea2715766e26c47c7589ec4fb6df9c76d123658fcd849ea43542bbae9\": container with ID starting with 3238b5bea2715766e26c47c7589ec4fb6df9c76d123658fcd849ea43542bbae9 not found: ID does not exist"
Sep 11 13:36:04.782225 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.782191    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"]
Sep 11 13:36:04.801176 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:04.801150    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae1d6b61-857fbff76b-qjsz8"]
Sep 11 13:36:05.315482 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:05.315445    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a16abcc-2abc-42a4-a687-e79bb7405e49" path="/var/lib/kubelet/pods/2a16abcc-2abc-42a4-a687-e79bb7405e49/volumes"
Sep 11 13:36:05.315852 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:05.315820    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64c1ad9e-57da-4d5f-9a2a-f7fdd6697173" path="/var/lib/kubelet/pods/64c1ad9e-57da-4d5f-9a2a-f7fdd6697173/volumes"
Sep 11 13:36:21.309739 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:21.309653    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:36:21.311433 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:21.311400    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:36:33.332402 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.332365    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"]
Sep 11 13:36:33.332871 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.332850    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64c1ad9e-57da-4d5f-9a2a-f7fdd6697173" containerName="revoke-keys"
Sep 11 13:36:33.332975 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.332876    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="64c1ad9e-57da-4d5f-9a2a-f7fdd6697173" containerName="revoke-keys"
Sep 11 13:36:33.332975 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.332917    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a16abcc-2abc-42a4-a687-e79bb7405e49" containerName="payload-pre-processing"
Sep 11 13:36:33.332975 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.332928    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a16abcc-2abc-42a4-a687-e79bb7405e49" containerName="payload-pre-processing"
Sep 11 13:36:33.333145 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.333059    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="64c1ad9e-57da-4d5f-9a2a-f7fdd6697173" containerName="revoke-keys"
Sep 11 13:36:33.333145 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.333075    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a16abcc-2abc-42a4-a687-e79bb7405e49" containerName="payload-pre-processing"
Sep 11 13:36:33.340899 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.340877    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.344936 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.344888    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-082f62\"/\"default-dockercfg-478jz\""
Sep 11 13:36:33.345059 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.344938    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-082f62\"/\"simulator-w0-082f62-kserve-self-signed-certs\""
Sep 11 13:36:33.345059 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.344894    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-082f62\"/\"kube-root-ca.crt\""
Sep 11 13:36:33.345153 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.344894    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-082f62\"/\"openshift-service-ca.crt\""
Sep 11 13:36:33.354719 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.354693    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"]
Sep 11 13:36:33.388133 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.388099    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-tmp-dir\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.388267 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.388226    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-home\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.388305 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.388279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrtsl\" (UniqueName: \"kubernetes.io/projected/139c5206-1ca9-4779-b2b0-4c34717323e0-kube-api-access-wrtsl\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.388355 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.388329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-dshm\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.388404 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.388351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-model-cache\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.388501 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.388481    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/139c5206-1ca9-4779-b2b0-4c34717323e0-tls-certs\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.489235 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.489193    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-dshm\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.489235 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.489239    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-model-cache\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.489503 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.489298    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/139c5206-1ca9-4779-b2b0-4c34717323e0-tls-certs\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.489503 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.489329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-tmp-dir\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.489503 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.489406    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-home\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.489503 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.489428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrtsl\" (UniqueName: \"kubernetes.io/projected/139c5206-1ca9-4779-b2b0-4c34717323e0-kube-api-access-wrtsl\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.489741 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.489712    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-model-cache\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.489810 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.489741    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-home\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.489810 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.489772    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-tmp-dir\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.491490 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.491465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-dshm\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.491817 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.491798    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/139c5206-1ca9-4779-b2b0-4c34717323e0-tls-certs\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.499172 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.499151    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrtsl\" (UniqueName: \"kubernetes.io/projected/139c5206-1ca9-4779-b2b0-4c34717323e0-kube-api-access-wrtsl\") pod \"simulator-w0-082f62-kserve-dd7b776b9-7xfmb\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") " pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.652441 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.652353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:33.934573 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:33.934547    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"]
Sep 11 13:36:33.936020 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:36:33.935992    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod139c5206_1ca9_4779_b2b0_4c34717323e0.slice/crio-6fd4fa4306224ff9b067225e9b82abd2dcd673c3662dcd61f916bf459e95ef53 WatchSource:0}: Error finding container 6fd4fa4306224ff9b067225e9b82abd2dcd673c3662dcd61f916bf459e95ef53: Status 404 returned error can't find the container with id 6fd4fa4306224ff9b067225e9b82abd2dcd673c3662dcd61f916bf459e95ef53
Sep 11 13:36:34.924255 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:34.924218    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb" event={"ID":"139c5206-1ca9-4779-b2b0-4c34717323e0","Type":"ContainerStarted","Data":"05314b9fbd6f74d8679d775148e5228825b29e4a20f80649516b59d4b659e4b7"}
Sep 11 13:36:34.924634 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:34.924264    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb" event={"ID":"139c5206-1ca9-4779-b2b0-4c34717323e0","Type":"ContainerStarted","Data":"6fd4fa4306224ff9b067225e9b82abd2dcd673c3662dcd61f916bf459e95ef53"}
Sep 11 13:36:34.958563 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:34.958513    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb" podStartSLOduration=2.958497989 podStartE2EDuration="2.958497989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:36:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:36:34.958437049 +0000 UTC m=+914.176974542" watchObservedRunningTime="2026-09-11 13:36:34.958497989 +0000 UTC m=+914.177035481"
Sep 11 13:36:37.473963 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:37.473927    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:36:38.789796 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:38.789736    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"]
Sep 11 13:36:38.793666 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:38.793647    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"
Sep 11 13:36:38.796449 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:38.796424    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-5d979b8c-dockercfg-zkr4x\""
Sep 11 13:36:38.796560 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:38.796468    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-5d979b8c\""
Sep 11 13:36:38.805584 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:38.805554    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"]
Sep 11 13:36:38.849120 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:38.849081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66a00397-e05e-4d91-b1c9-20852135553e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx\" (UID: \"66a00397-e05e-4d91-b1c9-20852135553e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"
Sep 11 13:36:38.849307 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:38.849195    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2pfq\" (UniqueName: \"kubernetes.io/projected/66a00397-e05e-4d91-b1c9-20852135553e-kube-api-access-k2pfq\") pod \"payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx\" (UID: \"66a00397-e05e-4d91-b1c9-20852135553e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"
Sep 11 13:36:38.949629 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:38.949598    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66a00397-e05e-4d91-b1c9-20852135553e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx\" (UID: \"66a00397-e05e-4d91-b1c9-20852135553e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"
Sep 11 13:36:38.949810 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:38.949674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2pfq\" (UniqueName: \"kubernetes.io/projected/66a00397-e05e-4d91-b1c9-20852135553e-kube-api-access-k2pfq\") pod \"payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx\" (UID: \"66a00397-e05e-4d91-b1c9-20852135553e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"
Sep 11 13:36:38.950359 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:38.950335    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66a00397-e05e-4d91-b1c9-20852135553e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx\" (UID: \"66a00397-e05e-4d91-b1c9-20852135553e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"
Sep 11 13:36:38.957958 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:38.957932    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2pfq\" (UniqueName: \"kubernetes.io/projected/66a00397-e05e-4d91-b1c9-20852135553e-kube-api-access-k2pfq\") pod \"payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx\" (UID: \"66a00397-e05e-4d91-b1c9-20852135553e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"
Sep 11 13:36:39.106048 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:39.106015    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"
Sep 11 13:36:39.379204 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:39.379175    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"]
Sep 11 13:36:39.380531 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:36:39.380503    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66a00397_e05e_4d91_b1c9_20852135553e.slice/crio-d9a420eadd82ca73b84c28aecdb8f7ce462f760c872dbc24a7ff515fb67a7e21 WatchSource:0}: Error finding container d9a420eadd82ca73b84c28aecdb8f7ce462f760c872dbc24a7ff515fb67a7e21: Status 404 returned error can't find the container with id d9a420eadd82ca73b84c28aecdb8f7ce462f760c872dbc24a7ff515fb67a7e21
Sep 11 13:36:39.955652 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:39.955616    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx" event={"ID":"66a00397-e05e-4d91-b1c9-20852135553e","Type":"ContainerStarted","Data":"9e70da5a10ca4c8b6c9e09ebd1804768a512f7cc675f8b59280199d1ac8ed394"}
Sep 11 13:36:39.956107 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:39.955680    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"
Sep 11 13:36:39.956107 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:39.955708    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx" event={"ID":"66a00397-e05e-4d91-b1c9-20852135553e","Type":"ContainerStarted","Data":"d9a420eadd82ca73b84c28aecdb8f7ce462f760c872dbc24a7ff515fb67a7e21"}
Sep 11 13:36:39.976602 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:39.976556    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx" podStartSLOduration=1.9765411130000001 podStartE2EDuration="1.976541113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:36:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:36:39.973081702 +0000 UTC m=+919.191619194" watchObservedRunningTime="2026-09-11 13:36:39.976541113 +0000 UTC m=+919.195078605"
Sep 11 13:36:43.652919 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:43.652873    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:43.653323 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:43.652962    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:43.665634 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:43.665603    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:43.989158 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:43.989079    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:36:44.253063 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.252978    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"]
Sep 11 13:36:44.256764 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.256747    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.260215 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.260196    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-f9f26e\"/\"kube-root-ca.crt\""
Sep 11 13:36:44.260452 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.260434    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-f9f26e\"/\"autores-model-f9f26e-kserve-self-signed-certs\""
Sep 11 13:36:44.260509 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.260467    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-f9f26e\"/\"openshift-service-ca.crt\""
Sep 11 13:36:44.261304 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.261288    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-f9f26e\"/\"default-dockercfg-k8ln9\""
Sep 11 13:36:44.274537 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.274505    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"]
Sep 11 13:36:44.299177 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.299144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtlmv\" (UniqueName: \"kubernetes.io/projected/7ea70e6b-a01d-43b1-92b2-d38e46244751-kube-api-access-mtlmv\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.299335 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.299190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ea70e6b-a01d-43b1-92b2-d38e46244751-tls-certs\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.299335 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.299279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-model-cache\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.299335 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.299326    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-home\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.299439 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.299358    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-tmp-dir\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.299439 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.299392    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-dshm\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.400772 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.400731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-model-cache\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.400997 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.400887    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-home\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.400997 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.400962    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-tmp-dir\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.401130 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.401007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-dshm\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.401130 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.401103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtlmv\" (UniqueName: \"kubernetes.io/projected/7ea70e6b-a01d-43b1-92b2-d38e46244751-kube-api-access-mtlmv\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.401249 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.401138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ea70e6b-a01d-43b1-92b2-d38e46244751-tls-certs\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.401249 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.401201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-model-cache\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.401331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.401261    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-home\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.401331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.401310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-tmp-dir\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.403472 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.403444    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-dshm\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.403688 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.403670    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ea70e6b-a01d-43b1-92b2-d38e46244751-tls-certs\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.411025 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.411005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtlmv\" (UniqueName: \"kubernetes.io/projected/7ea70e6b-a01d-43b1-92b2-d38e46244751-kube-api-access-mtlmv\") pod \"autores-model-f9f26e-kserve-54cf9999bc-2gthp\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") " pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.566185 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.566146    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:44.850722 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.850696    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"]
Sep 11 13:36:44.852133 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:36:44.852107    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ea70e6b_a01d_43b1_92b2_d38e46244751.slice/crio-7eedc50230b2f830c1e7e6d9a2b366fd2d732a7a1a9ee3337badcf09d3bcfdb3 WatchSource:0}: Error finding container 7eedc50230b2f830c1e7e6d9a2b366fd2d732a7a1a9ee3337badcf09d3bcfdb3: Status 404 returned error can't find the container with id 7eedc50230b2f830c1e7e6d9a2b366fd2d732a7a1a9ee3337badcf09d3bcfdb3
Sep 11 13:36:44.985397 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.985357    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp" event={"ID":"7ea70e6b-a01d-43b1-92b2-d38e46244751","Type":"ContainerStarted","Data":"eff3d5489de65d69bcaed0491699fcc8a96e8a4ff8c86fd8f8a02a879095b3f2"}
Sep 11 13:36:44.985397 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:44.985406    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp" event={"ID":"7ea70e6b-a01d-43b1-92b2-d38e46244751","Type":"ContainerStarted","Data":"7eedc50230b2f830c1e7e6d9a2b366fd2d732a7a1a9ee3337badcf09d3bcfdb3"}
Sep 11 13:36:45.007849 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:45.007798    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp" podStartSLOduration=1.007780869 podStartE2EDuration="1.007780869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:36:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:36:45.004390527 +0000 UTC m=+924.222928020" watchObservedRunningTime="2026-09-11 13:36:45.007780869 +0000 UTC m=+924.226318410"
Sep 11 13:36:50.963107 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:50.963073    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"
Sep 11 13:36:54.566848 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:54.566812    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:54.567404 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:54.566866    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:54.579240 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:54.579214    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:55.051815 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:55.051738    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:36:56.785723 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:36:56.785684    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:37:04.031826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:04.031792    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts"]
Sep 11 13:37:04.037331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:04.037291    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts"
Sep 11 13:37:04.041178 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:04.041141    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:37:04.041424 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:04.041158    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:37:04.045824 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:04.045798    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts"]
Sep 11 13:37:04.183135 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:04.183084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/002bca2f-71a6-454d-af29-70a4b548d760-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts\" (UID: \"002bca2f-71a6-454d-af29-70a4b548d760\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts"
Sep 11 13:37:04.284199 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:04.284105    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/002bca2f-71a6-454d-af29-70a4b548d760-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts\" (UID: \"002bca2f-71a6-454d-af29-70a4b548d760\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts"
Sep 11 13:37:04.284738 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:04.284716    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/002bca2f-71a6-454d-af29-70a4b548d760-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts\" (UID: \"002bca2f-71a6-454d-af29-70a4b548d760\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts"
Sep 11 13:37:04.350973 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:04.350936    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts"
Sep 11 13:37:04.625639 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:04.625415    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts"]
Sep 11 13:37:04.629539 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:37:04.629494    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod002bca2f_71a6_454d_af29_70a4b548d760.slice/crio-92774909c5ab55ea7c81e40dc7c5ac5fdde405910e53ca4e1c0c23b0638c3e75 WatchSource:0}: Error finding container 92774909c5ab55ea7c81e40dc7c5ac5fdde405910e53ca4e1c0c23b0638c3e75: Status 404 returned error can't find the container with id 92774909c5ab55ea7c81e40dc7c5ac5fdde405910e53ca4e1c0c23b0638c3e75
Sep 11 13:37:05.103416 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:05.103381    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts" event={"ID":"002bca2f-71a6-454d-af29-70a4b548d760","Type":"ContainerStarted","Data":"1ef232c88e0c78e0f7d4349488550de1ca92c34788426c592a297e06527c568f"}
Sep 11 13:37:05.103802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:05.103439    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts" event={"ID":"002bca2f-71a6-454d-af29-70a4b548d760","Type":"ContainerStarted","Data":"92774909c5ab55ea7c81e40dc7c5ac5fdde405910e53ca4e1c0c23b0638c3e75"}
Sep 11 13:37:06.111231 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:06.111195    2580 generic.go:356] "Generic (PLEG): container finished" podID="002bca2f-71a6-454d-af29-70a4b548d760" containerID="1ef232c88e0c78e0f7d4349488550de1ca92c34788426c592a297e06527c568f" exitCode=0
Sep 11 13:37:06.111649 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:06.111287    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts" event={"ID":"002bca2f-71a6-454d-af29-70a4b548d760","Type":"ContainerDied","Data":"1ef232c88e0c78e0f7d4349488550de1ca92c34788426c592a297e06527c568f"}
Sep 11 13:37:07.258452 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:07.258428    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts"
Sep 11 13:37:07.413947 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:07.413829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/002bca2f-71a6-454d-af29-70a4b548d760-maas-api-service-ca\") pod \"002bca2f-71a6-454d-af29-70a4b548d760\" (UID: \"002bca2f-71a6-454d-af29-70a4b548d760\") "
Sep 11 13:37:07.414290 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:07.414251    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/002bca2f-71a6-454d-af29-70a4b548d760-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "002bca2f-71a6-454d-af29-70a4b548d760" (UID: "002bca2f-71a6-454d-af29-70a4b548d760"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:07.514790 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:07.514742    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/002bca2f-71a6-454d-af29-70a4b548d760-maas-api-service-ca\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:37:08.124086 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:08.124061    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts"
Sep 11 13:37:08.124293 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:08.124103    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts" event={"ID":"002bca2f-71a6-454d-af29-70a4b548d760","Type":"ContainerDied","Data":"92774909c5ab55ea7c81e40dc7c5ac5fdde405910e53ca4e1c0c23b0638c3e75"}
Sep 11 13:37:08.124293 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:08.124147    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="92774909c5ab55ea7c81e40dc7c5ac5fdde405910e53ca4e1c0c23b0638c3e75"
Sep 11 13:37:14.832427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.832392    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"]
Sep 11 13:37:14.832813 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.832798    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="002bca2f-71a6-454d-af29-70a4b548d760" containerName="revoke-keys"
Sep 11 13:37:14.832858 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.832814    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="002bca2f-71a6-454d-af29-70a4b548d760" containerName="revoke-keys"
Sep 11 13:37:14.832928 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.832898    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="002bca2f-71a6-454d-af29-70a4b548d760" containerName="revoke-keys"
Sep 11 13:37:14.836122 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.836106    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.840325 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.840301    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-371c38\"/\"simulator-w4-371c38-kserve-self-signed-certs\""
Sep 11 13:37:14.840470 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.840348    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-371c38\"/\"kube-root-ca.crt\""
Sep 11 13:37:14.840470 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.840353    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-371c38\"/\"openshift-service-ca.crt\""
Sep 11 13:37:14.840470 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.840353    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-371c38\"/\"default-dockercfg-xvr7g\""
Sep 11 13:37:14.878764 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.878732    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"]
Sep 11 13:37:14.885363 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.885335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-tls-certs\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.885473 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.885381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-tmp-dir\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.885473 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.885448    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-dshm\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.885553 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.885518    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5gv4\" (UniqueName: \"kubernetes.io/projected/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-kube-api-access-z5gv4\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.885553 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.885546    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-home\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.885635 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.885566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-model-cache\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.986274 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.986226    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-tmp-dir\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.986477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.986293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-dshm\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.986477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.986343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5gv4\" (UniqueName: \"kubernetes.io/projected/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-kube-api-access-z5gv4\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.986477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.986373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-home\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.986477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.986398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-model-cache\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.986698 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.986484    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-tls-certs\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.986698 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.986647    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-tmp-dir\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.986806 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.986770    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-home\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.986868 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.986817    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-model-cache\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.988516 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.988493    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-dshm\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:14.988881 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:14.988863    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-tls-certs\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:15.011336 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:15.011307    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5gv4\" (UniqueName: \"kubernetes.io/projected/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-kube-api-access-z5gv4\") pod \"simulator-w4-371c38-kserve-68749b6cfd-87jwf\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") " pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:15.144997 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:15.144888    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:15.432063 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:15.432035    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"]
Sep 11 13:37:15.434853 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:37:15.434829    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c82572f_6798_4d50_9dc4_124b7f0bd1d9.slice/crio-a8469f36978aec2a954006ce52bfd253b379d3d4a4484c189700a3b3171cf85d WatchSource:0}: Error finding container a8469f36978aec2a954006ce52bfd253b379d3d4a4484c189700a3b3171cf85d: Status 404 returned error can't find the container with id a8469f36978aec2a954006ce52bfd253b379d3d4a4484c189700a3b3171cf85d
Sep 11 13:37:16.171625 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:16.171584    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf" event={"ID":"5c82572f-6798-4d50-9dc4-124b7f0bd1d9","Type":"ContainerStarted","Data":"38449096d2049d480c82fdd268cc053df4f8dc3d6e4a1da7c65729f2dc48ea42"}
Sep 11 13:37:16.171625 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:16.171627    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf" event={"ID":"5c82572f-6798-4d50-9dc4-124b7f0bd1d9","Type":"ContainerStarted","Data":"a8469f36978aec2a954006ce52bfd253b379d3d4a4484c189700a3b3171cf85d"}
Sep 11 13:37:16.221763 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:16.221707    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf" podStartSLOduration=2.221692055 podStartE2EDuration="2.221692055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:37:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:37:16.218125234 +0000 UTC m=+955.436662725" watchObservedRunningTime="2026-09-11 13:37:16.221692055 +0000 UTC m=+955.440229547"
Sep 11 13:37:16.444566 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:16.444489    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"]
Sep 11 13:37:16.444734 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:16.444708    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx" podUID="66a00397-e05e-4d91-b1c9-20852135553e" containerName="payload-pre-processing" containerID="cri-o://9e70da5a10ca4c8b6c9e09ebd1804768a512f7cc675f8b59280199d1ac8ed394" gracePeriod=30
Sep 11 13:37:16.711952 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:16.711875    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"
Sep 11 13:37:16.747275 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:16.747240    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts"]
Sep 11 13:37:16.751229 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:16.751200    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5d979b8c-mr5ts"]
Sep 11 13:37:16.800827 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:16.800794    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2pfq\" (UniqueName: \"kubernetes.io/projected/66a00397-e05e-4d91-b1c9-20852135553e-kube-api-access-k2pfq\") pod \"66a00397-e05e-4d91-b1c9-20852135553e\" (UID: \"66a00397-e05e-4d91-b1c9-20852135553e\") "
Sep 11 13:37:16.801055 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:16.800846    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66a00397-e05e-4d91-b1c9-20852135553e-plugins-config-volume\") pod \"66a00397-e05e-4d91-b1c9-20852135553e\" (UID: \"66a00397-e05e-4d91-b1c9-20852135553e\") "
Sep 11 13:37:16.801247 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:16.801220    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66a00397-e05e-4d91-b1c9-20852135553e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "66a00397-e05e-4d91-b1c9-20852135553e" (UID: "66a00397-e05e-4d91-b1c9-20852135553e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:16.803002 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:16.802970    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66a00397-e05e-4d91-b1c9-20852135553e-kube-api-access-k2pfq" (OuterVolumeSpecName: "kube-api-access-k2pfq") pod "66a00397-e05e-4d91-b1c9-20852135553e" (UID: "66a00397-e05e-4d91-b1c9-20852135553e"). InnerVolumeSpecName "kube-api-access-k2pfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:16.902188 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:16.902150    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2pfq\" (UniqueName: \"kubernetes.io/projected/66a00397-e05e-4d91-b1c9-20852135553e-kube-api-access-k2pfq\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:37:16.902188 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:16.902180    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66a00397-e05e-4d91-b1c9-20852135553e-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:37:17.179156 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:17.179121    2580 generic.go:356] "Generic (PLEG): container finished" podID="66a00397-e05e-4d91-b1c9-20852135553e" containerID="9e70da5a10ca4c8b6c9e09ebd1804768a512f7cc675f8b59280199d1ac8ed394" exitCode=0
Sep 11 13:37:17.179630 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:17.179167    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"
Sep 11 13:37:17.179630 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:17.179200    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx" event={"ID":"66a00397-e05e-4d91-b1c9-20852135553e","Type":"ContainerDied","Data":"9e70da5a10ca4c8b6c9e09ebd1804768a512f7cc675f8b59280199d1ac8ed394"}
Sep 11 13:37:17.179630 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:17.179255    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx" event={"ID":"66a00397-e05e-4d91-b1c9-20852135553e","Type":"ContainerDied","Data":"d9a420eadd82ca73b84c28aecdb8f7ce462f760c872dbc24a7ff515fb67a7e21"}
Sep 11 13:37:17.179630 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:17.179282    2580 scope.go:117] "RemoveContainer" containerID="9e70da5a10ca4c8b6c9e09ebd1804768a512f7cc675f8b59280199d1ac8ed394"
Sep 11 13:37:17.189110 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:17.189090    2580 scope.go:117] "RemoveContainer" containerID="9e70da5a10ca4c8b6c9e09ebd1804768a512f7cc675f8b59280199d1ac8ed394"
Sep 11 13:37:17.189397 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:37:17.189379    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e70da5a10ca4c8b6c9e09ebd1804768a512f7cc675f8b59280199d1ac8ed394\": container with ID starting with 9e70da5a10ca4c8b6c9e09ebd1804768a512f7cc675f8b59280199d1ac8ed394 not found: ID does not exist" containerID="9e70da5a10ca4c8b6c9e09ebd1804768a512f7cc675f8b59280199d1ac8ed394"
Sep 11 13:37:17.189451 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:17.189407    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e70da5a10ca4c8b6c9e09ebd1804768a512f7cc675f8b59280199d1ac8ed394"} err="failed to get container status \"9e70da5a10ca4c8b6c9e09ebd1804768a512f7cc675f8b59280199d1ac8ed394\": rpc error: code = NotFound desc = could not find container \"9e70da5a10ca4c8b6c9e09ebd1804768a512f7cc675f8b59280199d1ac8ed394\": container with ID starting with 9e70da5a10ca4c8b6c9e09ebd1804768a512f7cc675f8b59280199d1ac8ed394 not found: ID does not exist"
Sep 11 13:37:17.229071 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:17.229037    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"]
Sep 11 13:37:17.249265 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:17.249235    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-5d979b8c-7c6d64fff4-wp5nx"]
Sep 11 13:37:17.315738 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:17.315703    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="002bca2f-71a6-454d-af29-70a4b548d760" path="/var/lib/kubelet/pods/002bca2f-71a6-454d-af29-70a4b548d760/volumes"
Sep 11 13:37:17.316109 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:17.316095    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66a00397-e05e-4d91-b1c9-20852135553e" path="/var/lib/kubelet/pods/66a00397-e05e-4d91-b1c9-20852135553e/volumes"
Sep 11 13:37:23.279636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:23.279604    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:37:25.145284 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:25.145250    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:25.145710 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:25.145307    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:25.157538 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:25.157516    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:25.240014 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:25.239978    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:37:29.359546 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.359513    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"]
Sep 11 13:37:29.360162 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.359707    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9" podUID="2aa8689d-f67f-4f5e-a59a-8ce70236b9ab" containerName="payload-processing" containerID="cri-o://90b582c17abd2dac3c6b6da29721b7881cdf064e6f5208d98e376fec505c0466" gracePeriod=30
Sep 11 13:37:29.370702 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.370672    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"]
Sep 11 13:37:29.370986 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.370938    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc" podUID="36b3926c-a4e9-4aea-904c-67ab2e684ecd" containerName="payload-pre-processing" containerID="cri-o://9f0c5bd46c08aa282f359ad667011fa428acc79a9fb45724a3c102d40d2ca7c8" gracePeriod=30
Sep 11 13:37:29.659003 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.658980    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"
Sep 11 13:37:29.661566 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.661546    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"
Sep 11 13:37:29.727566 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.727530    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kt2xn\" (UniqueName: \"kubernetes.io/projected/36b3926c-a4e9-4aea-904c-67ab2e684ecd-kube-api-access-kt2xn\") pod \"36b3926c-a4e9-4aea-904c-67ab2e684ecd\" (UID: \"36b3926c-a4e9-4aea-904c-67ab2e684ecd\") "
Sep 11 13:37:29.727729 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.727573    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b3926c-a4e9-4aea-904c-67ab2e684ecd-plugins-config-volume\") pod \"36b3926c-a4e9-4aea-904c-67ab2e684ecd\" (UID: \"36b3926c-a4e9-4aea-904c-67ab2e684ecd\") "
Sep 11 13:37:29.727729 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.727706    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krsjm\" (UniqueName: \"kubernetes.io/projected/2aa8689d-f67f-4f5e-a59a-8ce70236b9ab-kube-api-access-krsjm\") pod \"2aa8689d-f67f-4f5e-a59a-8ce70236b9ab\" (UID: \"2aa8689d-f67f-4f5e-a59a-8ce70236b9ab\") "
Sep 11 13:37:29.727851 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.727756    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2aa8689d-f67f-4f5e-a59a-8ce70236b9ab-plugins-config-volume\") pod \"2aa8689d-f67f-4f5e-a59a-8ce70236b9ab\" (UID: \"2aa8689d-f67f-4f5e-a59a-8ce70236b9ab\") "
Sep 11 13:37:29.728042 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.728017    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36b3926c-a4e9-4aea-904c-67ab2e684ecd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "36b3926c-a4e9-4aea-904c-67ab2e684ecd" (UID: "36b3926c-a4e9-4aea-904c-67ab2e684ecd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:29.728152 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.728126    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36b3926c-a4e9-4aea-904c-67ab2e684ecd-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:37:29.728245 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.728219    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2aa8689d-f67f-4f5e-a59a-8ce70236b9ab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2aa8689d-f67f-4f5e-a59a-8ce70236b9ab" (UID: "2aa8689d-f67f-4f5e-a59a-8ce70236b9ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:29.729843 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.729820    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2aa8689d-f67f-4f5e-a59a-8ce70236b9ab-kube-api-access-krsjm" (OuterVolumeSpecName: "kube-api-access-krsjm") pod "2aa8689d-f67f-4f5e-a59a-8ce70236b9ab" (UID: "2aa8689d-f67f-4f5e-a59a-8ce70236b9ab"). InnerVolumeSpecName "kube-api-access-krsjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:29.729843 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.729832    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36b3926c-a4e9-4aea-904c-67ab2e684ecd-kube-api-access-kt2xn" (OuterVolumeSpecName: "kube-api-access-kt2xn") pod "36b3926c-a4e9-4aea-904c-67ab2e684ecd" (UID: "36b3926c-a4e9-4aea-904c-67ab2e684ecd"). InnerVolumeSpecName "kube-api-access-kt2xn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:29.829551 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.829516    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krsjm\" (UniqueName: \"kubernetes.io/projected/2aa8689d-f67f-4f5e-a59a-8ce70236b9ab-kube-api-access-krsjm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:37:29.829551 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.829546    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2aa8689d-f67f-4f5e-a59a-8ce70236b9ab-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:37:29.829753 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:29.829562    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kt2xn\" (UniqueName: \"kubernetes.io/projected/36b3926c-a4e9-4aea-904c-67ab2e684ecd-kube-api-access-kt2xn\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:37:30.261777 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.261741    2580 generic.go:356] "Generic (PLEG): container finished" podID="36b3926c-a4e9-4aea-904c-67ab2e684ecd" containerID="9f0c5bd46c08aa282f359ad667011fa428acc79a9fb45724a3c102d40d2ca7c8" exitCode=0
Sep 11 13:37:30.261996 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.261786    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"
Sep 11 13:37:30.261996 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.261834    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc" event={"ID":"36b3926c-a4e9-4aea-904c-67ab2e684ecd","Type":"ContainerDied","Data":"9f0c5bd46c08aa282f359ad667011fa428acc79a9fb45724a3c102d40d2ca7c8"}
Sep 11 13:37:30.261996 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.261883    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc" event={"ID":"36b3926c-a4e9-4aea-904c-67ab2e684ecd","Type":"ContainerDied","Data":"f6c47fb1c3dc8a7f9f5d2e55c11dcbe3add4e9f603b03d8280a6b24378654dec"}
Sep 11 13:37:30.261996 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.261920    2580 scope.go:117] "RemoveContainer" containerID="9f0c5bd46c08aa282f359ad667011fa428acc79a9fb45724a3c102d40d2ca7c8"
Sep 11 13:37:30.263310 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.263285    2580 generic.go:356] "Generic (PLEG): container finished" podID="2aa8689d-f67f-4f5e-a59a-8ce70236b9ab" containerID="90b582c17abd2dac3c6b6da29721b7881cdf064e6f5208d98e376fec505c0466" exitCode=0
Sep 11 13:37:30.263416 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.263320    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"
Sep 11 13:37:30.263416 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.263368    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9" event={"ID":"2aa8689d-f67f-4f5e-a59a-8ce70236b9ab","Type":"ContainerDied","Data":"90b582c17abd2dac3c6b6da29721b7881cdf064e6f5208d98e376fec505c0466"}
Sep 11 13:37:30.263517 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.263419    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9" event={"ID":"2aa8689d-f67f-4f5e-a59a-8ce70236b9ab","Type":"ContainerDied","Data":"b15664947e2a57587afd27abed2c3544bb0fe98ac19be19eca34e75443cdf9a3"}
Sep 11 13:37:30.272438 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.272417    2580 scope.go:117] "RemoveContainer" containerID="9f0c5bd46c08aa282f359ad667011fa428acc79a9fb45724a3c102d40d2ca7c8"
Sep 11 13:37:30.272775 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:37:30.272749    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f0c5bd46c08aa282f359ad667011fa428acc79a9fb45724a3c102d40d2ca7c8\": container with ID starting with 9f0c5bd46c08aa282f359ad667011fa428acc79a9fb45724a3c102d40d2ca7c8 not found: ID does not exist" containerID="9f0c5bd46c08aa282f359ad667011fa428acc79a9fb45724a3c102d40d2ca7c8"
Sep 11 13:37:30.272875 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.272786    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f0c5bd46c08aa282f359ad667011fa428acc79a9fb45724a3c102d40d2ca7c8"} err="failed to get container status \"9f0c5bd46c08aa282f359ad667011fa428acc79a9fb45724a3c102d40d2ca7c8\": rpc error: code = NotFound desc = could not find container \"9f0c5bd46c08aa282f359ad667011fa428acc79a9fb45724a3c102d40d2ca7c8\": container with ID starting with 9f0c5bd46c08aa282f359ad667011fa428acc79a9fb45724a3c102d40d2ca7c8 not found: ID does not exist"
Sep 11 13:37:30.272875 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.272808    2580 scope.go:117] "RemoveContainer" containerID="90b582c17abd2dac3c6b6da29721b7881cdf064e6f5208d98e376fec505c0466"
Sep 11 13:37:30.282087 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.282069    2580 scope.go:117] "RemoveContainer" containerID="90b582c17abd2dac3c6b6da29721b7881cdf064e6f5208d98e376fec505c0466"
Sep 11 13:37:30.282390 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:37:30.282372    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90b582c17abd2dac3c6b6da29721b7881cdf064e6f5208d98e376fec505c0466\": container with ID starting with 90b582c17abd2dac3c6b6da29721b7881cdf064e6f5208d98e376fec505c0466 not found: ID does not exist" containerID="90b582c17abd2dac3c6b6da29721b7881cdf064e6f5208d98e376fec505c0466"
Sep 11 13:37:30.282450 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.282397    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90b582c17abd2dac3c6b6da29721b7881cdf064e6f5208d98e376fec505c0466"} err="failed to get container status \"90b582c17abd2dac3c6b6da29721b7881cdf064e6f5208d98e376fec505c0466\": rpc error: code = NotFound desc = could not find container \"90b582c17abd2dac3c6b6da29721b7881cdf064e6f5208d98e376fec505c0466\": container with ID starting with 90b582c17abd2dac3c6b6da29721b7881cdf064e6f5208d98e376fec505c0466 not found: ID does not exist"
Sep 11 13:37:30.290850 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.290829    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"]
Sep 11 13:37:30.293646 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.293627    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f9f26e-5564c6ff98-k66zc"]
Sep 11 13:37:30.309989 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.309961    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"]
Sep 11 13:37:30.315722 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:30.315694    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f9f26e-666c574685-kr5d9"]
Sep 11 13:37:31.315979 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:31.315942    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2aa8689d-f67f-4f5e-a59a-8ce70236b9ab" path="/var/lib/kubelet/pods/2aa8689d-f67f-4f5e-a59a-8ce70236b9ab/volumes"
Sep 11 13:37:31.316367 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:31.316316    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36b3926c-a4e9-4aea-904c-67ab2e684ecd" path="/var/lib/kubelet/pods/36b3926c-a4e9-4aea-904c-67ab2e684ecd/volumes"
Sep 11 13:37:35.010932 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:35.010886    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"]
Sep 11 13:37:43.957405 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:43.957371    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:37:48.463384 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.463349    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"]
Sep 11 13:37:48.463816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.463801    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66a00397-e05e-4d91-b1c9-20852135553e" containerName="payload-pre-processing"
Sep 11 13:37:48.463816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.463817    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="66a00397-e05e-4d91-b1c9-20852135553e" containerName="payload-pre-processing"
Sep 11 13:37:48.463889 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.463836    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36b3926c-a4e9-4aea-904c-67ab2e684ecd" containerName="payload-pre-processing"
Sep 11 13:37:48.463889 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.463843    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="36b3926c-a4e9-4aea-904c-67ab2e684ecd" containerName="payload-pre-processing"
Sep 11 13:37:48.463889 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.463863    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2aa8689d-f67f-4f5e-a59a-8ce70236b9ab" containerName="payload-processing"
Sep 11 13:37:48.463889 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.463869    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2aa8689d-f67f-4f5e-a59a-8ce70236b9ab" containerName="payload-processing"
Sep 11 13:37:48.464055 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.463963    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="66a00397-e05e-4d91-b1c9-20852135553e" containerName="payload-pre-processing"
Sep 11 13:37:48.464055 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.463975    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="36b3926c-a4e9-4aea-904c-67ab2e684ecd" containerName="payload-pre-processing"
Sep 11 13:37:48.464055 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.463981    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2aa8689d-f67f-4f5e-a59a-8ce70236b9ab" containerName="payload-processing"
Sep 11 13:37:48.466798 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.466781    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.471087 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.471060    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b65337\"/\"openshift-service-ca.crt\""
Sep 11 13:37:48.472187 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.472165    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b65337\"/\"test-model-b65337-kserve-self-signed-certs\""
Sep 11 13:37:48.472286 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.472235    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b65337\"/\"default-dockercfg-cxmw2\""
Sep 11 13:37:48.472286 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.472235    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b65337\"/\"kube-root-ca.crt\""
Sep 11 13:37:48.480263 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.480237    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"]
Sep 11 13:37:48.615590 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.615562    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-tmp-dir\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.615590 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.615595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-tls-certs\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.615802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.615621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzdz8\" (UniqueName: \"kubernetes.io/projected/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-kube-api-access-dzdz8\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.615802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.615698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-home\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.615802 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.615798    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-dshm\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.615899 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.615814    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-model-cache\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.717271 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.717193    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-dshm\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.717271 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.717226    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-model-cache\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.717456 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.717272    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-tmp-dir\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.717456 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.717295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-tls-certs\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.717456 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.717327    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzdz8\" (UniqueName: \"kubernetes.io/projected/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-kube-api-access-dzdz8\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.717456 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.717354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-home\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.717694 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.717672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-model-cache\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.717763 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.717700    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-home\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.717823 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.717760    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-tmp-dir\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.719408 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.719387    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-dshm\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.719756 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.719740    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-tls-certs\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.726461 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.726440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzdz8\" (UniqueName: \"kubernetes.io/projected/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-kube-api-access-dzdz8\") pod \"test-model-b65337-kserve-86bb57bdf-wr6g9\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") " pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:48.776710 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:48.776676    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:49.061950 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:49.061895    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"]
Sep 11 13:37:49.062158 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:37:49.062138    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod082d8ee1_199b_42c8_8011_0ad1c2ecfaab.slice/crio-30f1791908549f45f7af1f9cea6eae473f7da39bb5ccc102fc354beec13e46f7 WatchSource:0}: Error finding container 30f1791908549f45f7af1f9cea6eae473f7da39bb5ccc102fc354beec13e46f7: Status 404 returned error can't find the container with id 30f1791908549f45f7af1f9cea6eae473f7da39bb5ccc102fc354beec13e46f7
Sep 11 13:37:49.383917 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:49.383861    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9" event={"ID":"082d8ee1-199b-42c8-8011-0ad1c2ecfaab","Type":"ContainerStarted","Data":"308816049d653e420d10fea66c238e8b302b7195b223c9efebb30ef24da23e54"}
Sep 11 13:37:49.384092 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:49.383928    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9" event={"ID":"082d8ee1-199b-42c8-8011-0ad1c2ecfaab","Type":"ContainerStarted","Data":"30f1791908549f45f7af1f9cea6eae473f7da39bb5ccc102fc354beec13e46f7"}
Sep 11 13:37:49.435271 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:49.435029    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9" podStartSLOduration=1.435003412 podStartE2EDuration="1.435003412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:37:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:37:49.426563627 +0000 UTC m=+988.645101120" watchObservedRunningTime="2026-09-11 13:37:49.435003412 +0000 UTC m=+988.653540906"
Sep 11 13:37:50.031578 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.031514    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp" podUID="7ea70e6b-a01d-43b1-92b2-d38e46244751" containerName="main" containerID="cri-o://eff3d5489de65d69bcaed0491699fcc8a96e8a4ff8c86fd8f8a02a879095b3f2" gracePeriod=45
Sep 11 13:37:50.284486 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.284425    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:37:50.390964 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.390934    2580 generic.go:356] "Generic (PLEG): container finished" podID="7ea70e6b-a01d-43b1-92b2-d38e46244751" containerID="eff3d5489de65d69bcaed0491699fcc8a96e8a4ff8c86fd8f8a02a879095b3f2" exitCode=0
Sep 11 13:37:50.391135 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.391024    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"
Sep 11 13:37:50.391135 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.391024    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp" event={"ID":"7ea70e6b-a01d-43b1-92b2-d38e46244751","Type":"ContainerDied","Data":"eff3d5489de65d69bcaed0491699fcc8a96e8a4ff8c86fd8f8a02a879095b3f2"}
Sep 11 13:37:50.391135 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.391072    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp" event={"ID":"7ea70e6b-a01d-43b1-92b2-d38e46244751","Type":"ContainerDied","Data":"7eedc50230b2f830c1e7e6d9a2b366fd2d732a7a1a9ee3337badcf09d3bcfdb3"}
Sep 11 13:37:50.391135 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.391093    2580 scope.go:117] "RemoveContainer" containerID="eff3d5489de65d69bcaed0491699fcc8a96e8a4ff8c86fd8f8a02a879095b3f2"
Sep 11 13:37:50.400111 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.400064    2580 scope.go:117] "RemoveContainer" containerID="eff3d5489de65d69bcaed0491699fcc8a96e8a4ff8c86fd8f8a02a879095b3f2"
Sep 11 13:37:50.400371 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:37:50.400352    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eff3d5489de65d69bcaed0491699fcc8a96e8a4ff8c86fd8f8a02a879095b3f2\": container with ID starting with eff3d5489de65d69bcaed0491699fcc8a96e8a4ff8c86fd8f8a02a879095b3f2 not found: ID does not exist" containerID="eff3d5489de65d69bcaed0491699fcc8a96e8a4ff8c86fd8f8a02a879095b3f2"
Sep 11 13:37:50.400421 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.400381    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eff3d5489de65d69bcaed0491699fcc8a96e8a4ff8c86fd8f8a02a879095b3f2"} err="failed to get container status \"eff3d5489de65d69bcaed0491699fcc8a96e8a4ff8c86fd8f8a02a879095b3f2\": rpc error: code = NotFound desc = could not find container \"eff3d5489de65d69bcaed0491699fcc8a96e8a4ff8c86fd8f8a02a879095b3f2\": container with ID starting with eff3d5489de65d69bcaed0491699fcc8a96e8a4ff8c86fd8f8a02a879095b3f2 not found: ID does not exist"
Sep 11 13:37:50.433372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.433342    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-model-cache\") pod \"7ea70e6b-a01d-43b1-92b2-d38e46244751\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") "
Sep 11 13:37:50.433512 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.433388    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-tmp-dir\") pod \"7ea70e6b-a01d-43b1-92b2-d38e46244751\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") "
Sep 11 13:37:50.433512 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.433424    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtlmv\" (UniqueName: \"kubernetes.io/projected/7ea70e6b-a01d-43b1-92b2-d38e46244751-kube-api-access-mtlmv\") pod \"7ea70e6b-a01d-43b1-92b2-d38e46244751\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") "
Sep 11 13:37:50.433512 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.433457    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-home\") pod \"7ea70e6b-a01d-43b1-92b2-d38e46244751\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") "
Sep 11 13:37:50.433661 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.433523    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-dshm\") pod \"7ea70e6b-a01d-43b1-92b2-d38e46244751\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") "
Sep 11 13:37:50.433661 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.433581    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ea70e6b-a01d-43b1-92b2-d38e46244751-tls-certs\") pod \"7ea70e6b-a01d-43b1-92b2-d38e46244751\" (UID: \"7ea70e6b-a01d-43b1-92b2-d38e46244751\") "
Sep 11 13:37:50.433661 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.433633    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-model-cache" (OuterVolumeSpecName: "model-cache") pod "7ea70e6b-a01d-43b1-92b2-d38e46244751" (UID: "7ea70e6b-a01d-43b1-92b2-d38e46244751"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:50.433826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.433724    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7ea70e6b-a01d-43b1-92b2-d38e46244751" (UID: "7ea70e6b-a01d-43b1-92b2-d38e46244751"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:50.433826 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.433803    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-home" (OuterVolumeSpecName: "home") pod "7ea70e6b-a01d-43b1-92b2-d38e46244751" (UID: "7ea70e6b-a01d-43b1-92b2-d38e46244751"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:50.433961 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.433936    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:37:50.434027 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.433967    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:37:50.434027 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.433981    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:37:50.435694 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.435674    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-dshm" (OuterVolumeSpecName: "dshm") pod "7ea70e6b-a01d-43b1-92b2-d38e46244751" (UID: "7ea70e6b-a01d-43b1-92b2-d38e46244751"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:50.435791 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.435715    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ea70e6b-a01d-43b1-92b2-d38e46244751-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7ea70e6b-a01d-43b1-92b2-d38e46244751" (UID: "7ea70e6b-a01d-43b1-92b2-d38e46244751"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:37:50.435791 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.435734    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea70e6b-a01d-43b1-92b2-d38e46244751-kube-api-access-mtlmv" (OuterVolumeSpecName: "kube-api-access-mtlmv") pod "7ea70e6b-a01d-43b1-92b2-d38e46244751" (UID: "7ea70e6b-a01d-43b1-92b2-d38e46244751"). InnerVolumeSpecName "kube-api-access-mtlmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:50.535099 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.535009    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtlmv\" (UniqueName: \"kubernetes.io/projected/7ea70e6b-a01d-43b1-92b2-d38e46244751-kube-api-access-mtlmv\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:37:50.535099 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.535046    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ea70e6b-a01d-43b1-92b2-d38e46244751-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:37:50.535099 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.535061    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ea70e6b-a01d-43b1-92b2-d38e46244751-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:37:50.745112 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.745077    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"]
Sep 11 13:37:50.761947 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:50.761915    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-f9f26e/autores-model-f9f26e-kserve-54cf9999bc-2gthp"]
Sep 11 13:37:51.283480 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:51.283446    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9"]
Sep 11 13:37:51.283868 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:51.283855    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea70e6b-a01d-43b1-92b2-d38e46244751" containerName="main"
Sep 11 13:37:51.283868 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:51.283869    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea70e6b-a01d-43b1-92b2-d38e46244751" containerName="main"
Sep 11 13:37:51.283965 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:51.283960    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ea70e6b-a01d-43b1-92b2-d38e46244751" containerName="main"
Sep 11 13:37:51.287895 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:51.287880    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9"
Sep 11 13:37:51.290758 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:51.290733    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:37:51.290858 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:51.290841    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:37:51.296652 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:51.296630    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9"]
Sep 11 13:37:51.316953 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:51.316896    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ea70e6b-a01d-43b1-92b2-d38e46244751" path="/var/lib/kubelet/pods/7ea70e6b-a01d-43b1-92b2-d38e46244751/volumes"
Sep 11 13:37:51.442211 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:51.442171    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d564ade6-0086-451d-a872-cb99098f0480-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9\" (UID: \"d564ade6-0086-451d-a872-cb99098f0480\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9"
Sep 11 13:37:51.542710 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:51.542628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d564ade6-0086-451d-a872-cb99098f0480-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9\" (UID: \"d564ade6-0086-451d-a872-cb99098f0480\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9"
Sep 11 13:37:51.543299 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:51.543280    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d564ade6-0086-451d-a872-cb99098f0480-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9\" (UID: \"d564ade6-0086-451d-a872-cb99098f0480\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9"
Sep 11 13:37:51.600482 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:51.600443    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9"
Sep 11 13:37:51.876059 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:51.876029    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9"]
Sep 11 13:37:51.878752 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:37:51.878710    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd564ade6_0086_451d_a872_cb99098f0480.slice/crio-eeafc635edb1912554124256506b8159994979d320cc0eecd308d3ae3779f93f WatchSource:0}: Error finding container eeafc635edb1912554124256506b8159994979d320cc0eecd308d3ae3779f93f: Status 404 returned error can't find the container with id eeafc635edb1912554124256506b8159994979d320cc0eecd308d3ae3779f93f
Sep 11 13:37:52.407766 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:52.407727    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9" event={"ID":"d564ade6-0086-451d-a872-cb99098f0480","Type":"ContainerStarted","Data":"93336ec45608f3d79c48bb4028e57789b1c5191252d04842e0908df53c26ec8e"}
Sep 11 13:37:52.407766 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:52.407785    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9" event={"ID":"d564ade6-0086-451d-a872-cb99098f0480","Type":"ContainerStarted","Data":"eeafc635edb1912554124256506b8159994979d320cc0eecd308d3ae3779f93f"}
Sep 11 13:37:52.435212 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:52.435118    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9" podStartSLOduration=1.435101464 podStartE2EDuration="1.435101464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:37:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:37:52.433670489 +0000 UTC m=+991.652207982" watchObservedRunningTime="2026-09-11 13:37:52.435101464 +0000 UTC m=+991.653639001"
Sep 11 13:37:53.421312 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:53.421279    2580 generic.go:356] "Generic (PLEG): container finished" podID="d564ade6-0086-451d-a872-cb99098f0480" containerID="93336ec45608f3d79c48bb4028e57789b1c5191252d04842e0908df53c26ec8e" exitCode=0
Sep 11 13:37:53.421693 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:53.421332    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9" event={"ID":"d564ade6-0086-451d-a872-cb99098f0480","Type":"ContainerDied","Data":"93336ec45608f3d79c48bb4028e57789b1c5191252d04842e0908df53c26ec8e"}
Sep 11 13:37:54.563408 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:54.563382    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9"
Sep 11 13:37:54.666226 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:54.666192    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d564ade6-0086-451d-a872-cb99098f0480-maas-api-service-ca\") pod \"d564ade6-0086-451d-a872-cb99098f0480\" (UID: \"d564ade6-0086-451d-a872-cb99098f0480\") "
Sep 11 13:37:54.666528 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:54.666503    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d564ade6-0086-451d-a872-cb99098f0480-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d564ade6-0086-451d-a872-cb99098f0480" (UID: "d564ade6-0086-451d-a872-cb99098f0480"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:54.767176 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:54.767088    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d564ade6-0086-451d-a872-cb99098f0480-maas-api-service-ca\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:37:55.436204 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:55.436166    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9" event={"ID":"d564ade6-0086-451d-a872-cb99098f0480","Type":"ContainerDied","Data":"eeafc635edb1912554124256506b8159994979d320cc0eecd308d3ae3779f93f"}
Sep 11 13:37:55.436204 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:55.436200    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9"
Sep 11 13:37:55.436417 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:55.436229    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eeafc635edb1912554124256506b8159994979d320cc0eecd308d3ae3779f93f"
Sep 11 13:37:58.370318 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:58.370275    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:37:58.777547 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:58.777442    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:58.777547 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:58.777503    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:58.791838 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:58.791805    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:37:59.476037 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:37:59.476004    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:38:03.147303 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:03.147266    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9"]
Sep 11 13:38:03.155607 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:03.152994    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eeb77b9d-c7ls9"]
Sep 11 13:38:03.316574 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:03.316536    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d564ade6-0086-451d-a872-cb99098f0480" path="/var/lib/kubelet/pods/d564ade6-0086-451d-a872-cb99098f0480/volumes"
Sep 11 13:38:13.164040 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.164001    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"]
Sep 11 13:38:13.164441 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.164395    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d564ade6-0086-451d-a872-cb99098f0480" containerName="revoke-keys"
Sep 11 13:38:13.164441 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.164408    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d564ade6-0086-451d-a872-cb99098f0480" containerName="revoke-keys"
Sep 11 13:38:13.164523 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.164480    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d564ade6-0086-451d-a872-cb99098f0480" containerName="revoke-keys"
Sep 11 13:38:13.167856 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.167838    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"
Sep 11 13:38:13.170591 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.170570    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-2a5fb372-dockercfg-ntt9x\""
Sep 11 13:38:13.170939 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.170923    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-2a5fb372\""
Sep 11 13:38:13.176743 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.176723    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"]
Sep 11 13:38:13.244017 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.243983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de29b53c-386b-4cec-9836-a0b2b5607b77-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx\" (UID: \"de29b53c-386b-4cec-9836-a0b2b5607b77\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"
Sep 11 13:38:13.244180 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.244045    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vst9\" (UniqueName: \"kubernetes.io/projected/de29b53c-386b-4cec-9836-a0b2b5607b77-kube-api-access-2vst9\") pod \"payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx\" (UID: \"de29b53c-386b-4cec-9836-a0b2b5607b77\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"
Sep 11 13:38:13.344793 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.344739    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de29b53c-386b-4cec-9836-a0b2b5607b77-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx\" (UID: \"de29b53c-386b-4cec-9836-a0b2b5607b77\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"
Sep 11 13:38:13.345027 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.344833    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vst9\" (UniqueName: \"kubernetes.io/projected/de29b53c-386b-4cec-9836-a0b2b5607b77-kube-api-access-2vst9\") pod \"payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx\" (UID: \"de29b53c-386b-4cec-9836-a0b2b5607b77\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"
Sep 11 13:38:13.345450 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.345429    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de29b53c-386b-4cec-9836-a0b2b5607b77-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx\" (UID: \"de29b53c-386b-4cec-9836-a0b2b5607b77\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"
Sep 11 13:38:13.354560 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.354526    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vst9\" (UniqueName: \"kubernetes.io/projected/de29b53c-386b-4cec-9836-a0b2b5607b77-kube-api-access-2vst9\") pod \"payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx\" (UID: \"de29b53c-386b-4cec-9836-a0b2b5607b77\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"
Sep 11 13:38:13.479307 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.479200    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"
Sep 11 13:38:13.755307 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:13.755282    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"]
Sep 11 13:38:13.757356 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:38:13.757328    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde29b53c_386b_4cec_9836_a0b2b5607b77.slice/crio-051bc2cfd51f3cf3299dc9ea4b4272b6fac19082bacfbdd5d71055219f3cc9fb WatchSource:0}: Error finding container 051bc2cfd51f3cf3299dc9ea4b4272b6fac19082bacfbdd5d71055219f3cc9fb: Status 404 returned error can't find the container with id 051bc2cfd51f3cf3299dc9ea4b4272b6fac19082bacfbdd5d71055219f3cc9fb
Sep 11 13:38:14.557874 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:14.557834    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx" event={"ID":"de29b53c-386b-4cec-9836-a0b2b5607b77","Type":"ContainerStarted","Data":"9969be69512bb8624bf4aa97fd78f5b88b6c86ca3df7f8aaa6d36a7730dc7aa5"}
Sep 11 13:38:14.558596 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:14.557897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx" event={"ID":"de29b53c-386b-4cec-9836-a0b2b5607b77","Type":"ContainerStarted","Data":"051bc2cfd51f3cf3299dc9ea4b4272b6fac19082bacfbdd5d71055219f3cc9fb"}
Sep 11 13:38:14.558596 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:14.557939    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"
Sep 11 13:38:14.580133 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:14.580086    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx" podStartSLOduration=1.5800708430000001 podStartE2EDuration="1.580070843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:38:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:38:14.578739691 +0000 UTC m=+1013.797277183" watchObservedRunningTime="2026-09-11 13:38:14.580070843 +0000 UTC m=+1013.798608325"
Sep 11 13:38:22.280804 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:22.280758    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:38:24.602048 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:24.602012    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"]
Sep 11 13:38:24.606943 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:24.606897    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"
Sep 11 13:38:24.609635 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:24.609614    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-5a0fa1\""
Sep 11 13:38:24.609733 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:24.609648    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-5a0fa1-dockercfg-phbdq\""
Sep 11 13:38:24.618649 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:24.618626    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"]
Sep 11 13:38:24.753041 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:24.753007    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00e0d1e1-e2d4-4e70-932a-ce95dd160f1d-plugins-config-volume\") pod \"payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk\" (UID: \"00e0d1e1-e2d4-4e70-932a-ce95dd160f1d\") " pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"
Sep 11 13:38:24.753206 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:24.753101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scs4d\" (UniqueName: \"kubernetes.io/projected/00e0d1e1-e2d4-4e70-932a-ce95dd160f1d-kube-api-access-scs4d\") pod \"payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk\" (UID: \"00e0d1e1-e2d4-4e70-932a-ce95dd160f1d\") " pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"
Sep 11 13:38:24.854097 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:24.853945    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00e0d1e1-e2d4-4e70-932a-ce95dd160f1d-plugins-config-volume\") pod \"payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk\" (UID: \"00e0d1e1-e2d4-4e70-932a-ce95dd160f1d\") " pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"
Sep 11 13:38:24.854227 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:24.854115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scs4d\" (UniqueName: \"kubernetes.io/projected/00e0d1e1-e2d4-4e70-932a-ce95dd160f1d-kube-api-access-scs4d\") pod \"payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk\" (UID: \"00e0d1e1-e2d4-4e70-932a-ce95dd160f1d\") " pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"
Sep 11 13:38:24.854650 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:24.854628    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00e0d1e1-e2d4-4e70-932a-ce95dd160f1d-plugins-config-volume\") pod \"payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk\" (UID: \"00e0d1e1-e2d4-4e70-932a-ce95dd160f1d\") " pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"
Sep 11 13:38:24.863404 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:24.863382    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scs4d\" (UniqueName: \"kubernetes.io/projected/00e0d1e1-e2d4-4e70-932a-ce95dd160f1d-kube-api-access-scs4d\") pod \"payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk\" (UID: \"00e0d1e1-e2d4-4e70-932a-ce95dd160f1d\") " pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"
Sep 11 13:38:24.919581 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:24.919545    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"
Sep 11 13:38:25.199788 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:25.196316    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"]
Sep 11 13:38:25.566562 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:25.566536    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"
Sep 11 13:38:25.624967 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:25.624928    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk" event={"ID":"00e0d1e1-e2d4-4e70-932a-ce95dd160f1d","Type":"ContainerStarted","Data":"f197653a5cf835306f108bce4bdad0d32f7a0429d2bd1718a3cbbebe9af11bca"}
Sep 11 13:38:25.625362 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:25.624987    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk" event={"ID":"00e0d1e1-e2d4-4e70-932a-ce95dd160f1d","Type":"ContainerStarted","Data":"1daa939612afd644e42bd8c32eec35457d20d2c749c6b13f50ac4a1bb9921bbc"}
Sep 11 13:38:25.625362 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:25.625060    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"
Sep 11 13:38:25.647816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:25.647767    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk" podStartSLOduration=1.647749896 podStartE2EDuration="1.647749896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:38:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:38:25.645258421 +0000 UTC m=+1024.863795925" watchObservedRunningTime="2026-09-11 13:38:25.647749896 +0000 UTC m=+1024.866287390"
Sep 11 13:38:36.633706 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:36.633674    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"
Sep 11 13:38:49.118803 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.118767    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"]
Sep 11 13:38:49.119394 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.119042    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx" podUID="de29b53c-386b-4cec-9836-a0b2b5607b77" containerName="payload-pre-processing" containerID="cri-o://9969be69512bb8624bf4aa97fd78f5b88b6c86ca3df7f8aaa6d36a7730dc7aa5" gracePeriod=30
Sep 11 13:38:49.364577 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.364551    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"
Sep 11 13:38:49.497739 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.497661    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de29b53c-386b-4cec-9836-a0b2b5607b77-plugins-config-volume\") pod \"de29b53c-386b-4cec-9836-a0b2b5607b77\" (UID: \"de29b53c-386b-4cec-9836-a0b2b5607b77\") "
Sep 11 13:38:49.497887 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.497744    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vst9\" (UniqueName: \"kubernetes.io/projected/de29b53c-386b-4cec-9836-a0b2b5607b77-kube-api-access-2vst9\") pod \"de29b53c-386b-4cec-9836-a0b2b5607b77\" (UID: \"de29b53c-386b-4cec-9836-a0b2b5607b77\") "
Sep 11 13:38:49.498125 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.498098    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de29b53c-386b-4cec-9836-a0b2b5607b77-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "de29b53c-386b-4cec-9836-a0b2b5607b77" (UID: "de29b53c-386b-4cec-9836-a0b2b5607b77"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:38:49.499994 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.499969    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de29b53c-386b-4cec-9836-a0b2b5607b77-kube-api-access-2vst9" (OuterVolumeSpecName: "kube-api-access-2vst9") pod "de29b53c-386b-4cec-9836-a0b2b5607b77" (UID: "de29b53c-386b-4cec-9836-a0b2b5607b77"). InnerVolumeSpecName "kube-api-access-2vst9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:49.598770 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.598738    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de29b53c-386b-4cec-9836-a0b2b5607b77-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:38:49.598770 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.598764    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vst9\" (UniqueName: \"kubernetes.io/projected/de29b53c-386b-4cec-9836-a0b2b5607b77-kube-api-access-2vst9\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:38:49.767869 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.767780    2580 generic.go:356] "Generic (PLEG): container finished" podID="de29b53c-386b-4cec-9836-a0b2b5607b77" containerID="9969be69512bb8624bf4aa97fd78f5b88b6c86ca3df7f8aaa6d36a7730dc7aa5" exitCode=0
Sep 11 13:38:49.767869 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.767819    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"
Sep 11 13:38:49.768081 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.767868    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx" event={"ID":"de29b53c-386b-4cec-9836-a0b2b5607b77","Type":"ContainerDied","Data":"9969be69512bb8624bf4aa97fd78f5b88b6c86ca3df7f8aaa6d36a7730dc7aa5"}
Sep 11 13:38:49.768081 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.767935    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx" event={"ID":"de29b53c-386b-4cec-9836-a0b2b5607b77","Type":"ContainerDied","Data":"051bc2cfd51f3cf3299dc9ea4b4272b6fac19082bacfbdd5d71055219f3cc9fb"}
Sep 11 13:38:49.768081 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.767961    2580 scope.go:117] "RemoveContainer" containerID="9969be69512bb8624bf4aa97fd78f5b88b6c86ca3df7f8aaa6d36a7730dc7aa5"
Sep 11 13:38:49.780010 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.779986    2580 scope.go:117] "RemoveContainer" containerID="9969be69512bb8624bf4aa97fd78f5b88b6c86ca3df7f8aaa6d36a7730dc7aa5"
Sep 11 13:38:49.780313 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:38:49.780289    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9969be69512bb8624bf4aa97fd78f5b88b6c86ca3df7f8aaa6d36a7730dc7aa5\": container with ID starting with 9969be69512bb8624bf4aa97fd78f5b88b6c86ca3df7f8aaa6d36a7730dc7aa5 not found: ID does not exist" containerID="9969be69512bb8624bf4aa97fd78f5b88b6c86ca3df7f8aaa6d36a7730dc7aa5"
Sep 11 13:38:49.780420 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.780321    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9969be69512bb8624bf4aa97fd78f5b88b6c86ca3df7f8aaa6d36a7730dc7aa5"} err="failed to get container status \"9969be69512bb8624bf4aa97fd78f5b88b6c86ca3df7f8aaa6d36a7730dc7aa5\": rpc error: code = NotFound desc = could not find container \"9969be69512bb8624bf4aa97fd78f5b88b6c86ca3df7f8aaa6d36a7730dc7aa5\": container with ID starting with 9969be69512bb8624bf4aa97fd78f5b88b6c86ca3df7f8aaa6d36a7730dc7aa5 not found: ID does not exist"
Sep 11 13:38:49.794331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.794306    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"]
Sep 11 13:38:49.800447 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.800424    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2a5fb372-7979c58b8b-z8nvx"]
Sep 11 13:38:49.970208 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:49.970140    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:38:51.316456 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:38:51.316425    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de29b53c-386b-4cec-9836-a0b2b5607b77" path="/var/lib/kubelet/pods/de29b53c-386b-4cec-9836-a0b2b5607b77/volumes"
Sep 11 13:39:07.558875 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:07.558834    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:39:09.563500 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.563468    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"]
Sep 11 13:39:09.563933 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.563896    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de29b53c-386b-4cec-9836-a0b2b5607b77" containerName="payload-pre-processing"
Sep 11 13:39:09.563991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.563935    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="de29b53c-386b-4cec-9836-a0b2b5607b77" containerName="payload-pre-processing"
Sep 11 13:39:09.564036 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.564009    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="de29b53c-386b-4cec-9836-a0b2b5607b77" containerName="payload-pre-processing"
Sep 11 13:39:09.568356 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.568338    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.572367 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.572343    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-082f62\"/\"premium-w0-082f62-kserve-self-signed-certs\""
Sep 11 13:39:09.578535 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.578512    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"]
Sep 11 13:39:09.695561 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.695529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-home\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.695754 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.695571    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsp49\" (UniqueName: \"kubernetes.io/projected/b4a4416d-224d-4415-a78c-047bb76e3fef-kube-api-access-gsp49\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.695754 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.695685    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-tmp-dir\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.695754 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.695720    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-model-cache\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.695951 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.695791    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-dshm\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.695951 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.695836    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4a4416d-224d-4415-a78c-047bb76e3fef-tls-certs\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.796488 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.796448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-dshm\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.796488 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.796492    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4a4416d-224d-4415-a78c-047bb76e3fef-tls-certs\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.796734 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.796616    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-home\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.796734 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.796676    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsp49\" (UniqueName: \"kubernetes.io/projected/b4a4416d-224d-4415-a78c-047bb76e3fef-kube-api-access-gsp49\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.796848 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.796797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-tmp-dir\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.796848 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.796834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-model-cache\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.797031 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.797009    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-home\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.797101 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.797081    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-tmp-dir\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.797222 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.797194    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-model-cache\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.798865 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.798849    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-dshm\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.799050 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.799034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4a4416d-224d-4415-a78c-047bb76e3fef-tls-certs\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.805163 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.805137    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsp49\" (UniqueName: \"kubernetes.io/projected/b4a4416d-224d-4415-a78c-047bb76e3fef-kube-api-access-gsp49\") pod \"premium-w0-082f62-kserve-7f466dd67b-sphgw\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") " pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:09.878416 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:09.878345    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:10.152941 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:10.152895    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"]
Sep 11 13:39:10.154898 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:39:10.154872    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4a4416d_224d_4415_a78c_047bb76e3fef.slice/crio-5fc5886990b2f246a9109db67a9aa62a8fc11cf939eeac69d5deefd46b818f55 WatchSource:0}: Error finding container 5fc5886990b2f246a9109db67a9aa62a8fc11cf939eeac69d5deefd46b818f55: Status 404 returned error can't find the container with id 5fc5886990b2f246a9109db67a9aa62a8fc11cf939eeac69d5deefd46b818f55
Sep 11 13:39:10.894733 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:10.894698    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw" event={"ID":"b4a4416d-224d-4415-a78c-047bb76e3fef","Type":"ContainerStarted","Data":"e5a6cf462e5c9632e54cc4e74f040cf4ee975d3e81ae7db42e823ae2c9b6a096"}
Sep 11 13:39:10.894733 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:10.894740    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw" event={"ID":"b4a4416d-224d-4415-a78c-047bb76e3fef","Type":"ContainerStarted","Data":"5fc5886990b2f246a9109db67a9aa62a8fc11cf939eeac69d5deefd46b818f55"}
Sep 11 13:39:10.926528 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:10.926474    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw" podStartSLOduration=1.926459651 podStartE2EDuration="1.926459651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:39:10.925409789 +0000 UTC m=+1070.143947293" watchObservedRunningTime="2026-09-11 13:39:10.926459651 +0000 UTC m=+1070.144997143"
Sep 11 13:39:19.879197 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:19.879102    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:19.879197 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:19.879170    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:19.891651 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:19.891614    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:19.959365 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:19.959334    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:39:25.148890 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:25.148852    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:39:44.512872 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:39:44.512838    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:40:10.996960 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:10.996919    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:40:23.418162 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.418125    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"]
Sep 11 13:40:23.421747 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.421730    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.426230 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.426211    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a649d3\"/\"kube-root-ca.crt\""
Sep 11 13:40:23.426700 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.426680    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a649d3\"/\"test-model-a649d3-kserve-self-signed-certs\""
Sep 11 13:40:23.426797 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.426716    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a649d3\"/\"openshift-service-ca.crt\""
Sep 11 13:40:23.426797 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.426724    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a649d3\"/\"default-dockercfg-2v76m\""
Sep 11 13:40:23.441355 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.441319    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"]
Sep 11 13:40:23.493993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.493961    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-tmp-dir\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.493993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.493994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftxvx\" (UniqueName: \"kubernetes.io/projected/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-kube-api-access-ftxvx\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.494205 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.494073    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-model-cache\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.494205 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.494109    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-dshm\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.494275 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.494223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-home\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.494275 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.494241    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-tls-certs\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.594768 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.594735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-model-cache\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.594768 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.594774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-dshm\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.595055 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.594850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-home\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.595055 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.594868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-tls-certs\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.595055 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.594929    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-tmp-dir\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.595055 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.594956    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftxvx\" (UniqueName: \"kubernetes.io/projected/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-kube-api-access-ftxvx\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.595273 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.595247    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-model-cache\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.595331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.595290    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-home\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.595381 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.595360    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-tmp-dir\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.597181 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.597155    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-dshm\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.597436 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.597419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-tls-certs\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.604728 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.604704    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftxvx\" (UniqueName: \"kubernetes.io/projected/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-kube-api-access-ftxvx\") pod \"test-model-a649d3-kserve-764787995-mfzhq\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") " pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:23.730949 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:23.730840    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:24.008250 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:24.008226    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"]
Sep 11 13:40:24.010299 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:40:24.010267    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a967bf2_a01d_4e3c_8b0d_cd9ab3c4428f.slice/crio-df69e23dee45c0689de50f31f22277679791298480f5970ce46459fcae9c328d WatchSource:0}: Error finding container df69e23dee45c0689de50f31f22277679791298480f5970ce46459fcae9c328d: Status 404 returned error can't find the container with id df69e23dee45c0689de50f31f22277679791298480f5970ce46459fcae9c328d
Sep 11 13:40:24.342884 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:24.342851    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq" event={"ID":"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f","Type":"ContainerStarted","Data":"a0a5421004ac491197b9ad039117042bcdeb07b7dff7d65cc2ae539dd54f9a5b"}
Sep 11 13:40:24.343059 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:24.342894    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq" event={"ID":"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f","Type":"ContainerStarted","Data":"df69e23dee45c0689de50f31f22277679791298480f5970ce46459fcae9c328d"}
Sep 11 13:40:24.369463 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:24.369413    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq" podStartSLOduration=1.369398645 podStartE2EDuration="1.369398645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:40:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:40:24.367492756 +0000 UTC m=+1143.586030247" watchObservedRunningTime="2026-09-11 13:40:24.369398645 +0000 UTC m=+1143.587936136"
Sep 11 13:40:33.731380 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:33.731341    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:33.731380 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:33.731390    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:33.743939 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:33.743897    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:34.416048 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:34.416021    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:40:41.497757 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:41.497720    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:40:49.779489 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:49.779449    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"]
Sep 11 13:40:49.779981 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:49.779753    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk" podUID="00e0d1e1-e2d4-4e70-932a-ce95dd160f1d" containerName="payload-processing" containerID="cri-o://f197653a5cf835306f108bce4bdad0d32f7a0429d2bd1718a3cbbebe9af11bca" gracePeriod=30
Sep 11 13:40:50.035279 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.035219    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"
Sep 11 13:40:50.140393 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.140361    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scs4d\" (UniqueName: \"kubernetes.io/projected/00e0d1e1-e2d4-4e70-932a-ce95dd160f1d-kube-api-access-scs4d\") pod \"00e0d1e1-e2d4-4e70-932a-ce95dd160f1d\" (UID: \"00e0d1e1-e2d4-4e70-932a-ce95dd160f1d\") "
Sep 11 13:40:50.140534 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.140454    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00e0d1e1-e2d4-4e70-932a-ce95dd160f1d-plugins-config-volume\") pod \"00e0d1e1-e2d4-4e70-932a-ce95dd160f1d\" (UID: \"00e0d1e1-e2d4-4e70-932a-ce95dd160f1d\") "
Sep 11 13:40:50.140874 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.140848    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00e0d1e1-e2d4-4e70-932a-ce95dd160f1d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "00e0d1e1-e2d4-4e70-932a-ce95dd160f1d" (UID: "00e0d1e1-e2d4-4e70-932a-ce95dd160f1d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:40:50.142433 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.142412    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00e0d1e1-e2d4-4e70-932a-ce95dd160f1d-kube-api-access-scs4d" (OuterVolumeSpecName: "kube-api-access-scs4d") pod "00e0d1e1-e2d4-4e70-932a-ce95dd160f1d" (UID: "00e0d1e1-e2d4-4e70-932a-ce95dd160f1d"). InnerVolumeSpecName "kube-api-access-scs4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:40:50.241581 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.241540    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00e0d1e1-e2d4-4e70-932a-ce95dd160f1d-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:40:50.241581 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.241574    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scs4d\" (UniqueName: \"kubernetes.io/projected/00e0d1e1-e2d4-4e70-932a-ce95dd160f1d-kube-api-access-scs4d\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:40:50.502572 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.502534    2580 generic.go:356] "Generic (PLEG): container finished" podID="00e0d1e1-e2d4-4e70-932a-ce95dd160f1d" containerID="f197653a5cf835306f108bce4bdad0d32f7a0429d2bd1718a3cbbebe9af11bca" exitCode=0
Sep 11 13:40:50.502572 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.502576    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"
Sep 11 13:40:50.502851 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.502620    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk" event={"ID":"00e0d1e1-e2d4-4e70-932a-ce95dd160f1d","Type":"ContainerDied","Data":"f197653a5cf835306f108bce4bdad0d32f7a0429d2bd1718a3cbbebe9af11bca"}
Sep 11 13:40:50.502851 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.502678    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk" event={"ID":"00e0d1e1-e2d4-4e70-932a-ce95dd160f1d","Type":"ContainerDied","Data":"1daa939612afd644e42bd8c32eec35457d20d2c749c6b13f50ac4a1bb9921bbc"}
Sep 11 13:40:50.502851 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.502706    2580 scope.go:117] "RemoveContainer" containerID="f197653a5cf835306f108bce4bdad0d32f7a0429d2bd1718a3cbbebe9af11bca"
Sep 11 13:40:50.513566 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.513549    2580 scope.go:117] "RemoveContainer" containerID="f197653a5cf835306f108bce4bdad0d32f7a0429d2bd1718a3cbbebe9af11bca"
Sep 11 13:40:50.513828 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:40:50.513805    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f197653a5cf835306f108bce4bdad0d32f7a0429d2bd1718a3cbbebe9af11bca\": container with ID starting with f197653a5cf835306f108bce4bdad0d32f7a0429d2bd1718a3cbbebe9af11bca not found: ID does not exist" containerID="f197653a5cf835306f108bce4bdad0d32f7a0429d2bd1718a3cbbebe9af11bca"
Sep 11 13:40:50.513940 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.513833    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f197653a5cf835306f108bce4bdad0d32f7a0429d2bd1718a3cbbebe9af11bca"} err="failed to get container status \"f197653a5cf835306f108bce4bdad0d32f7a0429d2bd1718a3cbbebe9af11bca\": rpc error: code = NotFound desc = could not find container \"f197653a5cf835306f108bce4bdad0d32f7a0429d2bd1718a3cbbebe9af11bca\": container with ID starting with f197653a5cf835306f108bce4bdad0d32f7a0429d2bd1718a3cbbebe9af11bca not found: ID does not exist"
Sep 11 13:40:50.533298 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.533275    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"]
Sep 11 13:40:50.539359 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:50.539337    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-5a0fa1-769654b6f-8tgxk"]
Sep 11 13:40:51.316527 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:40:51.316491    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00e0d1e1-e2d4-4e70-932a-ce95dd160f1d" path="/var/lib/kubelet/pods/00e0d1e1-e2d4-4e70-932a-ce95dd160f1d/volumes"
Sep 11 13:41:21.351698 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:21.351666    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:41:21.355177 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:21.355153    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:41:22.064331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:22.064292    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:41:31.538461 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.538427    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"]
Sep 11 13:41:31.538870 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.538855    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00e0d1e1-e2d4-4e70-932a-ce95dd160f1d" containerName="payload-processing"
Sep 11 13:41:31.538929 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.538873    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="00e0d1e1-e2d4-4e70-932a-ce95dd160f1d" containerName="payload-processing"
Sep 11 13:41:31.538970 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.538960    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="00e0d1e1-e2d4-4e70-932a-ce95dd160f1d" containerName="payload-processing"
Sep 11 13:41:31.543102 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.543080    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.546689 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.546660    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-105163\"/\"kube-root-ca.crt\""
Sep 11 13:41:31.546814 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.546703    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-105163\"/\"default-dockercfg-r9bzd\""
Sep 11 13:41:31.546814 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.546705    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-105163\"/\"noten-model-105163-kserve-self-signed-certs\""
Sep 11 13:41:31.546814 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.546705    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-105163\"/\"openshift-service-ca.crt\""
Sep 11 13:41:31.555041 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.555016    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"]
Sep 11 13:41:31.652444 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.652406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-dshm\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.652637 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.652455    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-model-cache\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.652637 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.652515    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-tmp-dir\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.652637 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.652600    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-home\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.652769 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.652674    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzvfr\" (UniqueName: \"kubernetes.io/projected/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-kube-api-access-lzvfr\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.652769 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.652707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-tls-certs\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.753198 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.753162    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-model-cache\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.753384 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.753208    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-tmp-dir\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.753384 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.753241    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-home\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.753384 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.753284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzvfr\" (UniqueName: \"kubernetes.io/projected/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-kube-api-access-lzvfr\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.753566 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.753400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-tls-certs\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.753566 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.753477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-dshm\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.753566 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.753553    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-model-cache\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.753717 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.753599    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-home\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.753717 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.753661    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-tmp-dir\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.755973 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.755945    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-dshm\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.756106 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.756020    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-tls-certs\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.765894 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.765866    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzvfr\" (UniqueName: \"kubernetes.io/projected/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-kube-api-access-lzvfr\") pod \"noten-model-105163-kserve-5fc85876b7-4tcwj\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") " pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:31.853506 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:31.853466    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:32.140442 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:32.140407    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"]
Sep 11 13:41:32.140932 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:41:32.140884    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36bbd205_45e4_4b5d_a795_b5ecc7b5a83b.slice/crio-da554bb882e39a056405572cd397691c99e88b57188c173674a0fda236852be5 WatchSource:0}: Error finding container da554bb882e39a056405572cd397691c99e88b57188c173674a0fda236852be5: Status 404 returned error can't find the container with id da554bb882e39a056405572cd397691c99e88b57188c173674a0fda236852be5
Sep 11 13:41:32.754027 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:32.753994    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj" event={"ID":"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b","Type":"ContainerStarted","Data":"e13c00ddd475d04ff0fe6e570373864b4ef0b63585ae898068fc874f91d13eda"}
Sep 11 13:41:32.754027 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:32.754034    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj" event={"ID":"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b","Type":"ContainerStarted","Data":"da554bb882e39a056405572cd397691c99e88b57188c173674a0fda236852be5"}
Sep 11 13:41:32.782300 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:32.782251    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj" podStartSLOduration=1.782235888 podStartE2EDuration="1.782235888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:41:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:41:32.781173724 +0000 UTC m=+1211.999711216" watchObservedRunningTime="2026-09-11 13:41:32.782235888 +0000 UTC m=+1212.000773380"
Sep 11 13:41:41.854601 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:41.854556    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:41.854601 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:41.854609    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:41.867416 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:41.867386    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:42.824719 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:42.824680    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:41:58.972015 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:41:58.971975    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:42:12.183876 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:12.183832    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"]
Sep 11 13:42:21.430170 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:21.430079    2580 scope.go:117] "RemoveContainer" containerID="bafb5958bc5e5f8b708fd15e20876464056bf80aa99fe6a4b5a750b39347a4c5"
Sep 11 13:42:27.001946 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.001897    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"]
Sep 11 13:42:27.006420 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.006402    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"
Sep 11 13:42:27.009206 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.009177    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-e52aaa-dockercfg-sqg5d\""
Sep 11 13:42:27.009362 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.009297    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-e52aaa\""
Sep 11 13:42:27.016934 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.016890    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"]
Sep 11 13:42:27.103591 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.103557    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx56w\" (UniqueName: \"kubernetes.io/projected/40658cad-e2d6-4d22-97f4-90a264ac2a8a-kube-api-access-rx56w\") pod \"payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4\" (UID: \"40658cad-e2d6-4d22-97f4-90a264ac2a8a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"
Sep 11 13:42:27.103772 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.103616    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40658cad-e2d6-4d22-97f4-90a264ac2a8a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4\" (UID: \"40658cad-e2d6-4d22-97f4-90a264ac2a8a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"
Sep 11 13:42:27.203333 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.203280    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj" podUID="36bbd205-45e4-4b5d-a795-b5ecc7b5a83b" containerName="main" containerID="cri-o://e13c00ddd475d04ff0fe6e570373864b4ef0b63585ae898068fc874f91d13eda" gracePeriod=45
Sep 11 13:42:27.205234 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.205203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx56w\" (UniqueName: \"kubernetes.io/projected/40658cad-e2d6-4d22-97f4-90a264ac2a8a-kube-api-access-rx56w\") pod \"payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4\" (UID: \"40658cad-e2d6-4d22-97f4-90a264ac2a8a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"
Sep 11 13:42:27.205337 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.205275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40658cad-e2d6-4d22-97f4-90a264ac2a8a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4\" (UID: \"40658cad-e2d6-4d22-97f4-90a264ac2a8a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"
Sep 11 13:42:27.206229 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.206202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40658cad-e2d6-4d22-97f4-90a264ac2a8a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4\" (UID: \"40658cad-e2d6-4d22-97f4-90a264ac2a8a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"
Sep 11 13:42:27.214964 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.214939    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx56w\" (UniqueName: \"kubernetes.io/projected/40658cad-e2d6-4d22-97f4-90a264ac2a8a-kube-api-access-rx56w\") pod \"payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4\" (UID: \"40658cad-e2d6-4d22-97f4-90a264ac2a8a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"
Sep 11 13:42:27.317933 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.317880    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"
Sep 11 13:42:27.494682 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.494657    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:42:27.597408 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.597378    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"]
Sep 11 13:42:27.598779 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:42:27.598754    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40658cad_e2d6_4d22_97f4_90a264ac2a8a.slice/crio-275f594d4fb70d5eb5e24be585f55e5274b0f0de203c91bad0093f60b1e35048 WatchSource:0}: Error finding container 275f594d4fb70d5eb5e24be585f55e5274b0f0de203c91bad0093f60b1e35048: Status 404 returned error can't find the container with id 275f594d4fb70d5eb5e24be585f55e5274b0f0de203c91bad0093f60b1e35048
Sep 11 13:42:27.608707 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.608686    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-home\") pod \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") "
Sep 11 13:42:27.608797 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.608718    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-dshm\") pod \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") "
Sep 11 13:42:27.608797 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.608740    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-model-cache\") pod \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") "
Sep 11 13:42:27.608797 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.608763    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-tls-certs\") pod \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") "
Sep 11 13:42:27.608933 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.608860    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-tmp-dir\") pod \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") "
Sep 11 13:42:27.608933 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.608889    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzvfr\" (UniqueName: \"kubernetes.io/projected/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-kube-api-access-lzvfr\") pod \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\" (UID: \"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b\") "
Sep 11 13:42:27.609042 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.608991    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-model-cache" (OuterVolumeSpecName: "model-cache") pod "36bbd205-45e4-4b5d-a795-b5ecc7b5a83b" (UID: "36bbd205-45e4-4b5d-a795-b5ecc7b5a83b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:42:27.609097 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.609004    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-home" (OuterVolumeSpecName: "home") pod "36bbd205-45e4-4b5d-a795-b5ecc7b5a83b" (UID: "36bbd205-45e4-4b5d-a795-b5ecc7b5a83b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:42:27.609251 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.609222    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "36bbd205-45e4-4b5d-a795-b5ecc7b5a83b" (UID: "36bbd205-45e4-4b5d-a795-b5ecc7b5a83b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:42:27.609373 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.609329    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:42:27.609373 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.609344    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:42:27.609373 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.609353    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:42:27.611078 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.611049    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-kube-api-access-lzvfr" (OuterVolumeSpecName: "kube-api-access-lzvfr") pod "36bbd205-45e4-4b5d-a795-b5ecc7b5a83b" (UID: "36bbd205-45e4-4b5d-a795-b5ecc7b5a83b"). InnerVolumeSpecName "kube-api-access-lzvfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:42:27.611163 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.611099    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-dshm" (OuterVolumeSpecName: "dshm") pod "36bbd205-45e4-4b5d-a795-b5ecc7b5a83b" (UID: "36bbd205-45e4-4b5d-a795-b5ecc7b5a83b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:42:27.611163 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.611108    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "36bbd205-45e4-4b5d-a795-b5ecc7b5a83b" (UID: "36bbd205-45e4-4b5d-a795-b5ecc7b5a83b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:42:27.710893 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.710863    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:42:27.710893 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.710890    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:42:27.711100 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:27.710916    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzvfr\" (UniqueName: \"kubernetes.io/projected/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b-kube-api-access-lzvfr\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:42:28.088387 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:28.088353    2580 generic.go:356] "Generic (PLEG): container finished" podID="36bbd205-45e4-4b5d-a795-b5ecc7b5a83b" containerID="e13c00ddd475d04ff0fe6e570373864b4ef0b63585ae898068fc874f91d13eda" exitCode=0
Sep 11 13:42:28.088855 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:28.088392    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"
Sep 11 13:42:28.088855 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:28.088439    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj" event={"ID":"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b","Type":"ContainerDied","Data":"e13c00ddd475d04ff0fe6e570373864b4ef0b63585ae898068fc874f91d13eda"}
Sep 11 13:42:28.088855 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:28.088488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj" event={"ID":"36bbd205-45e4-4b5d-a795-b5ecc7b5a83b","Type":"ContainerDied","Data":"da554bb882e39a056405572cd397691c99e88b57188c173674a0fda236852be5"}
Sep 11 13:42:28.088855 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:28.088513    2580 scope.go:117] "RemoveContainer" containerID="e13c00ddd475d04ff0fe6e570373864b4ef0b63585ae898068fc874f91d13eda"
Sep 11 13:42:28.090284 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:28.090260    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4" event={"ID":"40658cad-e2d6-4d22-97f4-90a264ac2a8a","Type":"ContainerStarted","Data":"0f79bd90da5059f62c7441834566231677d103bf5c2005bd6f1ea1349e196c99"}
Sep 11 13:42:28.090410 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:28.090314    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"
Sep 11 13:42:28.090410 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:28.090335    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4" event={"ID":"40658cad-e2d6-4d22-97f4-90a264ac2a8a","Type":"ContainerStarted","Data":"275f594d4fb70d5eb5e24be585f55e5274b0f0de203c91bad0093f60b1e35048"}
Sep 11 13:42:28.098045 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:28.098023    2580 scope.go:117] "RemoveContainer" containerID="e13c00ddd475d04ff0fe6e570373864b4ef0b63585ae898068fc874f91d13eda"
Sep 11 13:42:28.098374 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:28.098350    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e13c00ddd475d04ff0fe6e570373864b4ef0b63585ae898068fc874f91d13eda\": container with ID starting with e13c00ddd475d04ff0fe6e570373864b4ef0b63585ae898068fc874f91d13eda not found: ID does not exist" containerID="e13c00ddd475d04ff0fe6e570373864b4ef0b63585ae898068fc874f91d13eda"
Sep 11 13:42:28.098480 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:28.098383    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e13c00ddd475d04ff0fe6e570373864b4ef0b63585ae898068fc874f91d13eda"} err="failed to get container status \"e13c00ddd475d04ff0fe6e570373864b4ef0b63585ae898068fc874f91d13eda\": rpc error: code = NotFound desc = could not find container \"e13c00ddd475d04ff0fe6e570373864b4ef0b63585ae898068fc874f91d13eda\": container with ID starting with e13c00ddd475d04ff0fe6e570373864b4ef0b63585ae898068fc874f91d13eda not found: ID does not exist"
Sep 11 13:42:28.128273 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:28.128226    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4" podStartSLOduration=2.128210607 podStartE2EDuration="2.128210607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:42:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:42:28.126256489 +0000 UTC m=+1267.344793980" watchObservedRunningTime="2026-09-11 13:42:28.128210607 +0000 UTC m=+1267.346748099"
Sep 11 13:42:28.150981 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:28.150951    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"]
Sep 11 13:42:28.159678 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:28.159651    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-105163/noten-model-105163-kserve-5fc85876b7-4tcwj"]
Sep 11 13:42:29.316262 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:29.316230    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36bbd205-45e4-4b5d-a795-b5ecc7b5a83b" path="/var/lib/kubelet/pods/36bbd205-45e4-4b5d-a795-b5ecc7b5a83b/volumes"
Sep 11 13:42:30.193480 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:30.193439    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:42:39.100429 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:39.100395    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"
Sep 11 13:42:50.526833 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:50.526794    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7613-37437d"]
Sep 11 13:42:50.527238 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:50.527226    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36bbd205-45e4-4b5d-a795-b5ecc7b5a83b" containerName="main"
Sep 11 13:42:50.527238 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:50.527239    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="36bbd205-45e4-4b5d-a795-b5ecc7b5a83b" containerName="main"
Sep 11 13:42:50.527336 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:50.527329    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="36bbd205-45e4-4b5d-a795-b5ecc7b5a83b" containerName="main"
Sep 11 13:42:50.530338 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:50.530321    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-37437d"
Sep 11 13:42:50.533045 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:50.533008    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7ltrq\""
Sep 11 13:42:50.536302 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:50.536274    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-37437d"]
Sep 11 13:42:50.638274 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:50.638238    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jtsw\" (UniqueName: \"kubernetes.io/projected/d514e6a2-2c12-4b87-8046-51361aa2d303-kube-api-access-8jtsw\") pod \"test-curl-7613-37437d\" (UID: \"d514e6a2-2c12-4b87-8046-51361aa2d303\") " pod="opendatahub/test-curl-7613-37437d"
Sep 11 13:42:50.738886 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:50.738851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jtsw\" (UniqueName: \"kubernetes.io/projected/d514e6a2-2c12-4b87-8046-51361aa2d303-kube-api-access-8jtsw\") pod \"test-curl-7613-37437d\" (UID: \"d514e6a2-2c12-4b87-8046-51361aa2d303\") " pod="opendatahub/test-curl-7613-37437d"
Sep 11 13:42:50.747226 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:50.747198    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jtsw\" (UniqueName: \"kubernetes.io/projected/d514e6a2-2c12-4b87-8046-51361aa2d303-kube-api-access-8jtsw\") pod \"test-curl-7613-37437d\" (UID: \"d514e6a2-2c12-4b87-8046-51361aa2d303\") " pod="opendatahub/test-curl-7613-37437d"
Sep 11 13:42:50.841238 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:50.841210    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-37437d"
Sep 11 13:42:51.121144 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:51.121116    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-37437d"]
Sep 11 13:42:51.122397 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:42:51.122369    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd514e6a2_2c12_4b87_8046_51361aa2d303.slice/crio-6179277fd2a478b553da1cbb51ed23cdebafc7212f472b0671f9462f66d9dfba WatchSource:0}: Error finding container 6179277fd2a478b553da1cbb51ed23cdebafc7212f472b0671f9462f66d9dfba: Status 404 returned error can't find the container with id 6179277fd2a478b553da1cbb51ed23cdebafc7212f472b0671f9462f66d9dfba
Sep 11 13:42:51.239329 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:51.239279    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-37437d" event={"ID":"d514e6a2-2c12-4b87-8046-51361aa2d303","Type":"ContainerStarted","Data":"eb6e28c6f325a28403e921de9ad3dfd3c9bb37af44d378860e24d4715a84203a"}
Sep 11 13:42:51.239329 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:51.239334    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-37437d" event={"ID":"d514e6a2-2c12-4b87-8046-51361aa2d303","Type":"ContainerStarted","Data":"6179277fd2a478b553da1cbb51ed23cdebafc7212f472b0671f9462f66d9dfba"}
Sep 11 13:42:51.257058 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:51.256991    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7613-37437d" podStartSLOduration=1.256976158 podStartE2EDuration="1.256976158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:42:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:42:51.254305883 +0000 UTC m=+1290.472843376" watchObservedRunningTime="2026-09-11 13:42:51.256976158 +0000 UTC m=+1290.475513649"
Sep 11 13:42:51.871575 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:51.871527    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7613-37437d"]
Sep 11 13:42:51.875108 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:51.875079    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7613-37437d"]
Sep 11 13:42:51.985751 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:51.985716    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7613-a94226"]
Sep 11 13:42:51.986148 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:51.986134    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d514e6a2-2c12-4b87-8046-51361aa2d303" containerName="test-curl-7613-37437d"
Sep 11 13:42:51.986148 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:51.986148    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d514e6a2-2c12-4b87-8046-51361aa2d303" containerName="test-curl-7613-37437d"
Sep 11 13:42:51.986256 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:51.986233    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d514e6a2-2c12-4b87-8046-51361aa2d303" containerName="test-curl-7613-37437d"
Sep 11 13:42:51.989333 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:51.989317    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-a94226"
Sep 11 13:42:51.995727 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:51.995693    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-a94226"]
Sep 11 13:42:52.051105 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:52.051072    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tkpj\" (UniqueName: \"kubernetes.io/projected/8d6b589e-f7fc-43e3-b7a0-64b17ac418e5-kube-api-access-6tkpj\") pod \"test-curl-7613-a94226\" (UID: \"8d6b589e-f7fc-43e3-b7a0-64b17ac418e5\") " pod="opendatahub/test-curl-7613-a94226"
Sep 11 13:42:52.151787 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:52.151692    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tkpj\" (UniqueName: \"kubernetes.io/projected/8d6b589e-f7fc-43e3-b7a0-64b17ac418e5-kube-api-access-6tkpj\") pod \"test-curl-7613-a94226\" (UID: \"8d6b589e-f7fc-43e3-b7a0-64b17ac418e5\") " pod="opendatahub/test-curl-7613-a94226"
Sep 11 13:42:52.160332 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:52.160304    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tkpj\" (UniqueName: \"kubernetes.io/projected/8d6b589e-f7fc-43e3-b7a0-64b17ac418e5-kube-api-access-6tkpj\") pod \"test-curl-7613-a94226\" (UID: \"8d6b589e-f7fc-43e3-b7a0-64b17ac418e5\") " pod="opendatahub/test-curl-7613-a94226"
Sep 11 13:42:52.244859 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:52.244830    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7613-37437d" container="test-curl-7613-37437d"
Sep 11 13:42:52.300750 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:52.300716    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-a94226"
Sep 11 13:42:52.576462 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:52.576437    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-a94226"]
Sep 11 13:42:52.578075 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:42:52.578049    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d6b589e_f7fc_43e3_b7a0_64b17ac418e5.slice/crio-c5d18f4bbc8ba409df946fdf758187f25eb78ae81270cd7a890e56584879c923 WatchSource:0}: Error finding container c5d18f4bbc8ba409df946fdf758187f25eb78ae81270cd7a890e56584879c923: Status 404 returned error can't find the container with id c5d18f4bbc8ba409df946fdf758187f25eb78ae81270cd7a890e56584879c923
Sep 11 13:42:53.251561 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:53.251517    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-a94226" event={"ID":"8d6b589e-f7fc-43e3-b7a0-64b17ac418e5","Type":"ContainerStarted","Data":"91c7d0e29cfc8e868add9331aa193ac3a7c54dd1b31fb312579b55a3f28cd3a6"}
Sep 11 13:42:53.251561 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:53.251565    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-a94226" event={"ID":"8d6b589e-f7fc-43e3-b7a0-64b17ac418e5","Type":"ContainerStarted","Data":"c5d18f4bbc8ba409df946fdf758187f25eb78ae81270cd7a890e56584879c923"}
Sep 11 13:42:53.252125 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:53.251591    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7613-37437d" podUID="d514e6a2-2c12-4b87-8046-51361aa2d303" containerName="test-curl-7613-37437d" containerID="cri-o://eb6e28c6f325a28403e921de9ad3dfd3c9bb37af44d378860e24d4715a84203a" gracePeriod=2
Sep 11 13:42:53.268795 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:53.268752    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7613-a94226" podStartSLOduration=2.268737548 podStartE2EDuration="2.268737548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:42:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:42:53.26671845 +0000 UTC m=+1292.485255943" watchObservedRunningTime="2026-09-11 13:42:53.268737548 +0000 UTC m=+1292.487275040"
Sep 11 13:42:53.269024 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:53.268995    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-37437d\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="d514e6a2-2c12-4b87-8046-51361aa2d303" pod="opendatahub/test-curl-7613-37437d"
Sep 11 13:42:53.271202 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:53.271181    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-37437d\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="d514e6a2-2c12-4b87-8046-51361aa2d303" pod="opendatahub/test-curl-7613-37437d"
Sep 11 13:42:53.865978 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:53.865946    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7613-a94226"]
Sep 11 13:42:53.874570 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:53.874542    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7613-a94226"]
Sep 11 13:42:53.984299 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:53.984220    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7613-463ff3"]
Sep 11 13:42:53.984646 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:53.984633    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d6b589e-f7fc-43e3-b7a0-64b17ac418e5" containerName="test-curl-7613-a94226"
Sep 11 13:42:53.984688 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:53.984648    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d6b589e-f7fc-43e3-b7a0-64b17ac418e5" containerName="test-curl-7613-a94226"
Sep 11 13:42:53.984745 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:53.984736    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d6b589e-f7fc-43e3-b7a0-64b17ac418e5" containerName="test-curl-7613-a94226"
Sep 11 13:42:53.987951 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:53.987936    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-463ff3"
Sep 11 13:42:53.993819 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:53.993791    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-463ff3"]
Sep 11 13:42:54.073751 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:54.073718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69vjn\" (UniqueName: \"kubernetes.io/projected/2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778-kube-api-access-69vjn\") pod \"test-curl-7613-463ff3\" (UID: \"2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778\") " pod="opendatahub/test-curl-7613-463ff3"
Sep 11 13:42:54.174706 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:54.174668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69vjn\" (UniqueName: \"kubernetes.io/projected/2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778-kube-api-access-69vjn\") pod \"test-curl-7613-463ff3\" (UID: \"2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778\") " pod="opendatahub/test-curl-7613-463ff3"
Sep 11 13:42:54.182837 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:54.182810    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69vjn\" (UniqueName: \"kubernetes.io/projected/2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778-kube-api-access-69vjn\") pod \"test-curl-7613-463ff3\" (UID: \"2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778\") " pod="opendatahub/test-curl-7613-463ff3"
Sep 11 13:42:54.257536 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:54.257445    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7613-a94226" container="test-curl-7613-a94226"
Sep 11 13:42:54.298448 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:54.298416    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-463ff3"
Sep 11 13:42:54.573794 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:54.573766    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-463ff3"]
Sep 11 13:42:54.574858 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:42:54.574830    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e3eb0d2_eb9e_47c9_a6f7_09ef930ec778.slice/crio-bbb835da685021a711d0de8c3c1fea0af94ec4230ab15b8b7b60dceabe00e8ec WatchSource:0}: Error finding container bbb835da685021a711d0de8c3c1fea0af94ec4230ab15b8b7b60dceabe00e8ec: Status 404 returned error can't find the container with id bbb835da685021a711d0de8c3c1fea0af94ec4230ab15b8b7b60dceabe00e8ec
Sep 11 13:42:55.265619 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:55.265585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-463ff3" event={"ID":"2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778","Type":"ContainerStarted","Data":"0a680325ae30ff838e3b5b9fdf1b4583cd793c019588b33923a4f32ef5edfd59"}
Sep 11 13:42:55.266036 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:55.265630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-463ff3" event={"ID":"2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778","Type":"ContainerStarted","Data":"bbb835da685021a711d0de8c3c1fea0af94ec4230ab15b8b7b60dceabe00e8ec"}
Sep 11 13:42:55.266036 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:55.265864    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7613-a94226" podUID="8d6b589e-f7fc-43e3-b7a0-64b17ac418e5" containerName="test-curl-7613-a94226" containerID="cri-o://91c7d0e29cfc8e868add9331aa193ac3a7c54dd1b31fb312579b55a3f28cd3a6" gracePeriod=2
Sep 11 13:42:55.283913 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:55.283849    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7613-463ff3" podStartSLOduration=2.283835048 podStartE2EDuration="2.283835048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:42:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:42:55.282568703 +0000 UTC m=+1294.501106196" watchObservedRunningTime="2026-09-11 13:42:55.283835048 +0000 UTC m=+1294.502372568"
Sep 11 13:42:55.285286 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:55.285252    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-a94226\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="8d6b589e-f7fc-43e3-b7a0-64b17ac418e5" pod="opendatahub/test-curl-7613-a94226"
Sep 11 13:42:55.431103 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:55.431078    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-37437d"
Sep 11 13:42:55.486602 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:55.486571    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jtsw\" (UniqueName: \"kubernetes.io/projected/d514e6a2-2c12-4b87-8046-51361aa2d303-kube-api-access-8jtsw\") pod \"d514e6a2-2c12-4b87-8046-51361aa2d303\" (UID: \"d514e6a2-2c12-4b87-8046-51361aa2d303\") "
Sep 11 13:42:55.488695 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:55.488661    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d514e6a2-2c12-4b87-8046-51361aa2d303-kube-api-access-8jtsw" (OuterVolumeSpecName: "kube-api-access-8jtsw") pod "d514e6a2-2c12-4b87-8046-51361aa2d303" (UID: "d514e6a2-2c12-4b87-8046-51361aa2d303"). InnerVolumeSpecName "kube-api-access-8jtsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:42:55.588279 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:55.588244    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jtsw\" (UniqueName: \"kubernetes.io/projected/d514e6a2-2c12-4b87-8046-51361aa2d303-kube-api-access-8jtsw\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:42:55.873115 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:55.873026    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7613-463ff3"]
Sep 11 13:42:55.891893 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:55.891865    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7613-463ff3"]
Sep 11 13:42:56.014265 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.014234    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7613-a6a0f2"]
Sep 11 13:42:56.014669 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.014657    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778" containerName="test-curl-7613-463ff3"
Sep 11 13:42:56.014713 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.014671    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778" containerName="test-curl-7613-463ff3"
Sep 11 13:42:56.014755 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.014746    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778" containerName="test-curl-7613-463ff3"
Sep 11 13:42:56.017798 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.017783    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-a6a0f2"
Sep 11 13:42:56.027395 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.027323    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-a6a0f2"]
Sep 11 13:42:56.092592 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.092559    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4m58\" (UniqueName: \"kubernetes.io/projected/a6ea6332-fa6f-40cd-8293-d7edef89a7c1-kube-api-access-p4m58\") pod \"test-curl-7613-a6a0f2\" (UID: \"a6ea6332-fa6f-40cd-8293-d7edef89a7c1\") " pod="opendatahub/test-curl-7613-a6a0f2"
Sep 11 13:42:56.193734 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.193644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4m58\" (UniqueName: \"kubernetes.io/projected/a6ea6332-fa6f-40cd-8293-d7edef89a7c1-kube-api-access-p4m58\") pod \"test-curl-7613-a6a0f2\" (UID: \"a6ea6332-fa6f-40cd-8293-d7edef89a7c1\") " pod="opendatahub/test-curl-7613-a6a0f2"
Sep 11 13:42:56.202236 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.202205    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4m58\" (UniqueName: \"kubernetes.io/projected/a6ea6332-fa6f-40cd-8293-d7edef89a7c1-kube-api-access-p4m58\") pod \"test-curl-7613-a6a0f2\" (UID: \"a6ea6332-fa6f-40cd-8293-d7edef89a7c1\") " pod="opendatahub/test-curl-7613-a6a0f2"
Sep 11 13:42:56.271885 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.271848    2580 generic.go:356] "Generic (PLEG): container finished" podID="d514e6a2-2c12-4b87-8046-51361aa2d303" containerID="eb6e28c6f325a28403e921de9ad3dfd3c9bb37af44d378860e24d4715a84203a" exitCode=137
Sep 11 13:42:56.272301 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.271898    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-37437d"
Sep 11 13:42:56.272301 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.271947    2580 scope.go:117] "RemoveContainer" containerID="eb6e28c6f325a28403e921de9ad3dfd3c9bb37af44d378860e24d4715a84203a"
Sep 11 13:42:56.272301 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:56.272196    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7613-463ff3" container="test-curl-7613-463ff3"
Sep 11 13:42:56.281433 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.281418    2580 scope.go:117] "RemoveContainer" containerID="eb6e28c6f325a28403e921de9ad3dfd3c9bb37af44d378860e24d4715a84203a"
Sep 11 13:42:56.281698 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:56.281681    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb6e28c6f325a28403e921de9ad3dfd3c9bb37af44d378860e24d4715a84203a\": container with ID starting with eb6e28c6f325a28403e921de9ad3dfd3c9bb37af44d378860e24d4715a84203a not found: ID does not exist" containerID="eb6e28c6f325a28403e921de9ad3dfd3c9bb37af44d378860e24d4715a84203a"
Sep 11 13:42:56.281736 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.281707    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb6e28c6f325a28403e921de9ad3dfd3c9bb37af44d378860e24d4715a84203a"} err="failed to get container status \"eb6e28c6f325a28403e921de9ad3dfd3c9bb37af44d378860e24d4715a84203a\": rpc error: code = NotFound desc = could not find container \"eb6e28c6f325a28403e921de9ad3dfd3c9bb37af44d378860e24d4715a84203a\": container with ID starting with eb6e28c6f325a28403e921de9ad3dfd3c9bb37af44d378860e24d4715a84203a not found: ID does not exist"
Sep 11 13:42:56.282982 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:56.282960    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-37437d\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="d514e6a2-2c12-4b87-8046-51361aa2d303" pod="opendatahub/test-curl-7613-37437d"
Sep 11 13:42:56.328327 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.328305    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-a6a0f2"
Sep 11 13:42:56.616678 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:56.616649    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-a6a0f2"]
Sep 11 13:42:56.618030 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:42:56.618004    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6ea6332_fa6f_40cd_8293_d7edef89a7c1.slice/crio-4d3109c754895d932bd5bb38493c8adb7edb4068ebdc9f6f2a2b91d212a5fb23 WatchSource:0}: Error finding container 4d3109c754895d932bd5bb38493c8adb7edb4068ebdc9f6f2a2b91d212a5fb23: Status 404 returned error can't find the container with id 4d3109c754895d932bd5bb38493c8adb7edb4068ebdc9f6f2a2b91d212a5fb23
Sep 11 13:42:57.279599 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:57.279570    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-a6a0f2" event={"ID":"a6ea6332-fa6f-40cd-8293-d7edef89a7c1","Type":"ContainerStarted","Data":"51b188a4818b517a4cd63231613ce4799bd0e337029f6614083501113ed6617f"}
Sep 11 13:42:57.279948 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:57.279613    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-a6a0f2" event={"ID":"a6ea6332-fa6f-40cd-8293-d7edef89a7c1","Type":"ContainerStarted","Data":"4d3109c754895d932bd5bb38493c8adb7edb4068ebdc9f6f2a2b91d212a5fb23"}
Sep 11 13:42:57.280893 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:57.280850    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7613-463ff3" podUID="2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778" containerName="test-curl-7613-463ff3" containerID="cri-o://0a680325ae30ff838e3b5b9fdf1b4583cd793c019588b33923a4f32ef5edfd59" gracePeriod=2
Sep 11 13:42:57.295097 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:57.295053    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7613-a6a0f2" podStartSLOduration=2.295038403 podStartE2EDuration="2.295038403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:42:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:42:57.293592297 +0000 UTC m=+1296.512129788" watchObservedRunningTime="2026-09-11 13:42:57.295038403 +0000 UTC m=+1296.513575894"
Sep 11 13:42:57.295565 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:57.295537    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-37437d\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="d514e6a2-2c12-4b87-8046-51361aa2d303" pod="opendatahub/test-curl-7613-37437d"
Sep 11 13:42:57.297484 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:57.297461    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-463ff3\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778" pod="opendatahub/test-curl-7613-463ff3"
Sep 11 13:42:57.299262 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:57.299238    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-37437d\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="d514e6a2-2c12-4b87-8046-51361aa2d303" pod="opendatahub/test-curl-7613-37437d"
Sep 11 13:42:57.317126 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:57.317104    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d514e6a2-2c12-4b87-8046-51361aa2d303" path="/var/lib/kubelet/pods/d514e6a2-2c12-4b87-8046-51361aa2d303/volumes"
Sep 11 13:42:57.447096 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:57.447070    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-a94226"
Sep 11 13:42:57.506287 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:57.506251    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tkpj\" (UniqueName: \"kubernetes.io/projected/8d6b589e-f7fc-43e3-b7a0-64b17ac418e5-kube-api-access-6tkpj\") pod \"8d6b589e-f7fc-43e3-b7a0-64b17ac418e5\" (UID: \"8d6b589e-f7fc-43e3-b7a0-64b17ac418e5\") "
Sep 11 13:42:57.508241 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:57.508197    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d6b589e-f7fc-43e3-b7a0-64b17ac418e5-kube-api-access-6tkpj" (OuterVolumeSpecName: "kube-api-access-6tkpj") pod "8d6b589e-f7fc-43e3-b7a0-64b17ac418e5" (UID: "8d6b589e-f7fc-43e3-b7a0-64b17ac418e5"). InnerVolumeSpecName "kube-api-access-6tkpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:42:57.607462 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:57.607430    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tkpj\" (UniqueName: \"kubernetes.io/projected/8d6b589e-f7fc-43e3-b7a0-64b17ac418e5-kube-api-access-6tkpj\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:42:57.863216 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:57.863126    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7613-a6a0f2"]
Sep 11 13:42:57.867817 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:57.867789    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7613-a6a0f2"]
Sep 11 13:42:58.288034 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:58.287938    2580 generic.go:356] "Generic (PLEG): container finished" podID="8d6b589e-f7fc-43e3-b7a0-64b17ac418e5" containerID="91c7d0e29cfc8e868add9331aa193ac3a7c54dd1b31fb312579b55a3f28cd3a6" exitCode=137
Sep 11 13:42:58.288034 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:58.288020    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-a94226"
Sep 11 13:42:58.288504 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:58.288079    2580 scope.go:117] "RemoveContainer" containerID="91c7d0e29cfc8e868add9331aa193ac3a7c54dd1b31fb312579b55a3f28cd3a6"
Sep 11 13:42:58.288504 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:58.288326    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7613-a6a0f2" container="test-curl-7613-a6a0f2"
Sep 11 13:42:58.297591 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:58.297572    2580 scope.go:117] "RemoveContainer" containerID="91c7d0e29cfc8e868add9331aa193ac3a7c54dd1b31fb312579b55a3f28cd3a6"
Sep 11 13:42:58.297850 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:58.297834    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91c7d0e29cfc8e868add9331aa193ac3a7c54dd1b31fb312579b55a3f28cd3a6\": container with ID starting with 91c7d0e29cfc8e868add9331aa193ac3a7c54dd1b31fb312579b55a3f28cd3a6 not found: ID does not exist" containerID="91c7d0e29cfc8e868add9331aa193ac3a7c54dd1b31fb312579b55a3f28cd3a6"
Sep 11 13:42:58.297895 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:58.297859    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91c7d0e29cfc8e868add9331aa193ac3a7c54dd1b31fb312579b55a3f28cd3a6"} err="failed to get container status \"91c7d0e29cfc8e868add9331aa193ac3a7c54dd1b31fb312579b55a3f28cd3a6\": rpc error: code = NotFound desc = could not find container \"91c7d0e29cfc8e868add9331aa193ac3a7c54dd1b31fb312579b55a3f28cd3a6\": container with ID starting with 91c7d0e29cfc8e868add9331aa193ac3a7c54dd1b31fb312579b55a3f28cd3a6 not found: ID does not exist"
Sep 11 13:42:58.302022 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:58.301999    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-a94226\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="8d6b589e-f7fc-43e3-b7a0-64b17ac418e5" pod="opendatahub/test-curl-7613-a94226"
Sep 11 13:42:59.295920 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:59.295852    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7613-a6a0f2" podUID="a6ea6332-fa6f-40cd-8293-d7edef89a7c1" containerName="test-curl-7613-a6a0f2" containerID="cri-o://51b188a4818b517a4cd63231613ce4799bd0e337029f6614083501113ed6617f" gracePeriod=2
Sep 11 13:42:59.302407 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:59.302376    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-a94226\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="8d6b589e-f7fc-43e3-b7a0-64b17ac418e5" pod="opendatahub/test-curl-7613-a94226"
Sep 11 13:42:59.305002 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:42:59.304960    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-a6a0f2\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="a6ea6332-fa6f-40cd-8293-d7edef89a7c1" pod="opendatahub/test-curl-7613-a6a0f2"
Sep 11 13:42:59.317592 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:59.317556    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d6b589e-f7fc-43e3-b7a0-64b17ac418e5" path="/var/lib/kubelet/pods/8d6b589e-f7fc-43e3-b7a0-64b17ac418e5/volumes"
Sep 11 13:42:59.432639 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:59.432616    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-463ff3"
Sep 11 13:42:59.524729 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:59.524695    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69vjn\" (UniqueName: \"kubernetes.io/projected/2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778-kube-api-access-69vjn\") pod \"2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778\" (UID: \"2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778\") "
Sep 11 13:42:59.526871 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:59.526844    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778-kube-api-access-69vjn" (OuterVolumeSpecName: "kube-api-access-69vjn") pod "2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778" (UID: "2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778"). InnerVolumeSpecName "kube-api-access-69vjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:42:59.625915 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:42:59.625880    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69vjn\" (UniqueName: \"kubernetes.io/projected/2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778-kube-api-access-69vjn\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:43:00.138543 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:00.138504    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:43:00.303886 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:00.303851    2580 generic.go:356] "Generic (PLEG): container finished" podID="2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778" containerID="0a680325ae30ff838e3b5b9fdf1b4583cd793c019588b33923a4f32ef5edfd59" exitCode=137
Sep 11 13:43:00.304259 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:00.303895    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-463ff3"
Sep 11 13:43:00.304259 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:00.303964    2580 scope.go:117] "RemoveContainer" containerID="0a680325ae30ff838e3b5b9fdf1b4583cd793c019588b33923a4f32ef5edfd59"
Sep 11 13:43:00.316201 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:00.316174    2580 scope.go:117] "RemoveContainer" containerID="0a680325ae30ff838e3b5b9fdf1b4583cd793c019588b33923a4f32ef5edfd59"
Sep 11 13:43:00.316528 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:43:00.316500    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a680325ae30ff838e3b5b9fdf1b4583cd793c019588b33923a4f32ef5edfd59\": container with ID starting with 0a680325ae30ff838e3b5b9fdf1b4583cd793c019588b33923a4f32ef5edfd59 not found: ID does not exist" containerID="0a680325ae30ff838e3b5b9fdf1b4583cd793c019588b33923a4f32ef5edfd59"
Sep 11 13:43:00.316620 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:00.316540    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a680325ae30ff838e3b5b9fdf1b4583cd793c019588b33923a4f32ef5edfd59"} err="failed to get container status \"0a680325ae30ff838e3b5b9fdf1b4583cd793c019588b33923a4f32ef5edfd59\": rpc error: code = NotFound desc = could not find container \"0a680325ae30ff838e3b5b9fdf1b4583cd793c019588b33923a4f32ef5edfd59\": container with ID starting with 0a680325ae30ff838e3b5b9fdf1b4583cd793c019588b33923a4f32ef5edfd59 not found: ID does not exist"
Sep 11 13:43:00.318433 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:43:00.318406    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-463ff3\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778" pod="opendatahub/test-curl-7613-463ff3"
Sep 11 13:43:01.315836 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:43:01.315807    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-463ff3\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778" pod="opendatahub/test-curl-7613-463ff3"
Sep 11 13:43:01.316771 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:01.316749    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778" path="/var/lib/kubelet/pods/2e3eb0d2-eb9e-47c9-a6f7-09ef930ec778/volumes"
Sep 11 13:43:01.489289 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:01.489264    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-a6a0f2"
Sep 11 13:43:01.492429 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:43:01.492403    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-a6a0f2\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="a6ea6332-fa6f-40cd-8293-d7edef89a7c1" pod="opendatahub/test-curl-7613-a6a0f2"
Sep 11 13:43:01.545799 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:01.545761    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4m58\" (UniqueName: \"kubernetes.io/projected/a6ea6332-fa6f-40cd-8293-d7edef89a7c1-kube-api-access-p4m58\") pod \"a6ea6332-fa6f-40cd-8293-d7edef89a7c1\" (UID: \"a6ea6332-fa6f-40cd-8293-d7edef89a7c1\") "
Sep 11 13:43:01.547793 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:01.547765    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6ea6332-fa6f-40cd-8293-d7edef89a7c1-kube-api-access-p4m58" (OuterVolumeSpecName: "kube-api-access-p4m58") pod "a6ea6332-fa6f-40cd-8293-d7edef89a7c1" (UID: "a6ea6332-fa6f-40cd-8293-d7edef89a7c1"). InnerVolumeSpecName "kube-api-access-p4m58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:43:01.646653 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:01.646615    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4m58\" (UniqueName: \"kubernetes.io/projected/a6ea6332-fa6f-40cd-8293-d7edef89a7c1-kube-api-access-p4m58\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:43:02.319272 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:02.319237    2580 generic.go:356] "Generic (PLEG): container finished" podID="a6ea6332-fa6f-40cd-8293-d7edef89a7c1" containerID="51b188a4818b517a4cd63231613ce4799bd0e337029f6614083501113ed6617f" exitCode=137
Sep 11 13:43:02.319714 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:02.319283    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-a6a0f2"
Sep 11 13:43:02.319714 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:02.319339    2580 scope.go:117] "RemoveContainer" containerID="51b188a4818b517a4cd63231613ce4799bd0e337029f6614083501113ed6617f"
Sep 11 13:43:02.322342 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:43:02.322317    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-a6a0f2\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="a6ea6332-fa6f-40cd-8293-d7edef89a7c1" pod="opendatahub/test-curl-7613-a6a0f2"
Sep 11 13:43:02.329720 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:02.329702    2580 scope.go:117] "RemoveContainer" containerID="51b188a4818b517a4cd63231613ce4799bd0e337029f6614083501113ed6617f"
Sep 11 13:43:02.330043 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:43:02.330025    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51b188a4818b517a4cd63231613ce4799bd0e337029f6614083501113ed6617f\": container with ID starting with 51b188a4818b517a4cd63231613ce4799bd0e337029f6614083501113ed6617f not found: ID does not exist" containerID="51b188a4818b517a4cd63231613ce4799bd0e337029f6614083501113ed6617f"
Sep 11 13:43:02.330113 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:02.330054    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51b188a4818b517a4cd63231613ce4799bd0e337029f6614083501113ed6617f"} err="failed to get container status \"51b188a4818b517a4cd63231613ce4799bd0e337029f6614083501113ed6617f\": rpc error: code = NotFound desc = could not find container \"51b188a4818b517a4cd63231613ce4799bd0e337029f6614083501113ed6617f\": container with ID starting with 51b188a4818b517a4cd63231613ce4799bd0e337029f6614083501113ed6617f not found: ID does not exist"
Sep 11 13:43:02.330709 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:43:02.330689    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-a6a0f2\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="a6ea6332-fa6f-40cd-8293-d7edef89a7c1" pod="opendatahub/test-curl-7613-a6a0f2"
Sep 11 13:43:03.316811 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:03.316776    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6ea6332-fa6f-40cd-8293-d7edef89a7c1" path="/var/lib/kubelet/pods/a6ea6332-fa6f-40cd-8293-d7edef89a7c1/volumes"
Sep 11 13:43:10.760182 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:10.760146    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"]
Sep 11 13:43:10.760563 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:10.760552    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6ea6332-fa6f-40cd-8293-d7edef89a7c1" containerName="test-curl-7613-a6a0f2"
Sep 11 13:43:10.760613 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:10.760565    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6ea6332-fa6f-40cd-8293-d7edef89a7c1" containerName="test-curl-7613-a6a0f2"
Sep 11 13:43:10.760657 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:10.760647    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6ea6332-fa6f-40cd-8293-d7edef89a7c1" containerName="test-curl-7613-a6a0f2"
Sep 11 13:43:10.765297 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:10.765275    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"
Sep 11 13:43:10.769170 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:10.769146    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c0f589fb-dockercfg-5tmpw\""
Sep 11 13:43:10.769288 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:10.769146    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c0f589fb\""
Sep 11 13:43:10.786255 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:10.786227    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"]
Sep 11 13:43:10.936367 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:10.936334    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvzm5\" (UniqueName: \"kubernetes.io/projected/7fc6cfad-c77d-49cf-b6f1-fb4e4466a335-kube-api-access-lvzm5\") pod \"payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl\" (UID: \"7fc6cfad-c77d-49cf-b6f1-fb4e4466a335\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"
Sep 11 13:43:10.936567 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:10.936473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fc6cfad-c77d-49cf-b6f1-fb4e4466a335-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl\" (UID: \"7fc6cfad-c77d-49cf-b6f1-fb4e4466a335\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"
Sep 11 13:43:11.037355 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:11.037261    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvzm5\" (UniqueName: \"kubernetes.io/projected/7fc6cfad-c77d-49cf-b6f1-fb4e4466a335-kube-api-access-lvzm5\") pod \"payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl\" (UID: \"7fc6cfad-c77d-49cf-b6f1-fb4e4466a335\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"
Sep 11 13:43:11.037534 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:11.037419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fc6cfad-c77d-49cf-b6f1-fb4e4466a335-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl\" (UID: \"7fc6cfad-c77d-49cf-b6f1-fb4e4466a335\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"
Sep 11 13:43:11.038144 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:11.038126    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fc6cfad-c77d-49cf-b6f1-fb4e4466a335-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl\" (UID: \"7fc6cfad-c77d-49cf-b6f1-fb4e4466a335\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"
Sep 11 13:43:11.055565 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:11.055532    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvzm5\" (UniqueName: \"kubernetes.io/projected/7fc6cfad-c77d-49cf-b6f1-fb4e4466a335-kube-api-access-lvzm5\") pod \"payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl\" (UID: \"7fc6cfad-c77d-49cf-b6f1-fb4e4466a335\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"
Sep 11 13:43:11.075337 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:11.075312    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"
Sep 11 13:43:11.394388 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:43:11.394359    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fc6cfad_c77d_49cf_b6f1_fb4e4466a335.slice/crio-ae4633f879263a3a1063f53046ce81bae113a0304996a5b61e968f434cebd52b WatchSource:0}: Error finding container ae4633f879263a3a1063f53046ce81bae113a0304996a5b61e968f434cebd52b: Status 404 returned error can't find the container with id ae4633f879263a3a1063f53046ce81bae113a0304996a5b61e968f434cebd52b
Sep 11 13:43:11.395462 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:11.395437    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"]
Sep 11 13:43:12.379613 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:12.379572    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl" event={"ID":"7fc6cfad-c77d-49cf-b6f1-fb4e4466a335","Type":"ContainerStarted","Data":"f400c0f1e521f41fe8f6b2e90bb321cc4d76185dcba60e8c79abca0fc810feca"}
Sep 11 13:43:12.380012 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:12.379637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl" event={"ID":"7fc6cfad-c77d-49cf-b6f1-fb4e4466a335","Type":"ContainerStarted","Data":"ae4633f879263a3a1063f53046ce81bae113a0304996a5b61e968f434cebd52b"}
Sep 11 13:43:12.380012 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:12.379670    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"
Sep 11 13:43:12.414300 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:12.414251    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl" podStartSLOduration=2.4142357 podStartE2EDuration="2.4142357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:43:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:43:12.414117057 +0000 UTC m=+1311.632654574" watchObservedRunningTime="2026-09-11 13:43:12.4142357 +0000 UTC m=+1311.632773193"
Sep 11 13:43:13.748350 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:13.748318    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"]
Sep 11 13:43:17.605657 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.605618    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"]
Sep 11 13:43:17.609296 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.609279    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.615466 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.615437    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-082f62\"/\"distinct-w0-082f62-kserve-self-signed-certs\""
Sep 11 13:43:17.624026 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.624003    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"]
Sep 11 13:43:17.703536 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.703498    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrkff\" (UniqueName: \"kubernetes.io/projected/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-kube-api-access-rrkff\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.703710 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.703615    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-model-cache\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.703710 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.703651    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-dshm\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.703710 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.703703    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-tls-certs\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.703863 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.703746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-tmp-dir\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.703863 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.703847    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-home\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.804594 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.804553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-model-cache\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.804594 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.804595    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-dshm\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.804820 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.804634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-tls-certs\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.804820 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.804656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-tmp-dir\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.804820 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.804684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-home\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.804820 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.804713    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrkff\" (UniqueName: \"kubernetes.io/projected/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-kube-api-access-rrkff\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.805270 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.805232    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-tmp-dir\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.806736 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.805552    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-model-cache\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.806736 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.805559    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-home\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.812981 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.808083    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-dshm\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.812981 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.808419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-tls-certs\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.815318 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.815292    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrkff\" (UniqueName: \"kubernetes.io/projected/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-kube-api-access-rrkff\") pod \"distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:17.918108 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:17.918008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:18.211495 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:18.211462    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"]
Sep 11 13:43:18.212718 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:43:18.212692    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c9ce5ed_23fe_4712_8dfb_09f1179cdfec.slice/crio-1e4b029fa050fa10b7847a56fcb4035d9a440a6d920be82542d0a2116038b196 WatchSource:0}: Error finding container 1e4b029fa050fa10b7847a56fcb4035d9a440a6d920be82542d0a2116038b196: Status 404 returned error can't find the container with id 1e4b029fa050fa10b7847a56fcb4035d9a440a6d920be82542d0a2116038b196
Sep 11 13:43:18.419246 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:18.419212    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv" event={"ID":"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec","Type":"ContainerStarted","Data":"4b6fdfa704c79b823e4e31d69b209c728c7ceb408bc8e6a96d6074eb21d5be16"}
Sep 11 13:43:18.419246 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:18.419252    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv" event={"ID":"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec","Type":"ContainerStarted","Data":"1e4b029fa050fa10b7847a56fcb4035d9a440a6d920be82542d0a2116038b196"}
Sep 11 13:43:18.443095 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:18.443040    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv" podStartSLOduration=1.443023267 podStartE2EDuration="1.443023267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:43:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:43:18.441180033 +0000 UTC m=+1317.659717524" watchObservedRunningTime="2026-09-11 13:43:18.443023267 +0000 UTC m=+1317.661560753"
Sep 11 13:43:21.453597 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:21.453567    2580 scope.go:117] "RemoveContainer" containerID="1ef232c88e0c78e0f7d4349488550de1ca92c34788426c592a297e06527c568f"
Sep 11 13:43:23.387727 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:23.387693    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"
Sep 11 13:43:27.338819 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:27.338775    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"]
Sep 11 13:43:27.918956 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:27.918896    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:27.919114 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:27.918975    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:27.931922 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:27.931878    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:28.513355 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:28.513316    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:43:28.576690 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:28.576646    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:43:28.767767 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:28.767666    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9" podUID="082d8ee1-199b-42c8-8011-0ad1c2ecfaab" containerName="main" containerID="cri-o://308816049d653e420d10fea66c238e8b302b7195b223c9efebb30ef24da23e54" gracePeriod=45
Sep 11 13:43:29.088633 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.088606    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:43:29.226320 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.226290    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-home\") pod \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") "
Sep 11 13:43:29.226487 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.226344    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzdz8\" (UniqueName: \"kubernetes.io/projected/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-kube-api-access-dzdz8\") pod \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") "
Sep 11 13:43:29.226487 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.226369    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-tls-certs\") pod \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") "
Sep 11 13:43:29.226487 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.226394    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-model-cache\") pod \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") "
Sep 11 13:43:29.226487 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.226453    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-dshm\") pod \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") "
Sep 11 13:43:29.226739 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.226489    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-tmp-dir\") pod \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\" (UID: \"082d8ee1-199b-42c8-8011-0ad1c2ecfaab\") "
Sep 11 13:43:29.226739 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.226596    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-home" (OuterVolumeSpecName: "home") pod "082d8ee1-199b-42c8-8011-0ad1c2ecfaab" (UID: "082d8ee1-199b-42c8-8011-0ad1c2ecfaab"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:29.226739 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.226688    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-model-cache" (OuterVolumeSpecName: "model-cache") pod "082d8ee1-199b-42c8-8011-0ad1c2ecfaab" (UID: "082d8ee1-199b-42c8-8011-0ad1c2ecfaab"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:29.226930 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.226861    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "082d8ee1-199b-42c8-8011-0ad1c2ecfaab" (UID: "082d8ee1-199b-42c8-8011-0ad1c2ecfaab"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:29.226988 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.226976    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:43:29.227038 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.226991    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:43:29.227038 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.227002    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:43:29.228796 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.228769    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "082d8ee1-199b-42c8-8011-0ad1c2ecfaab" (UID: "082d8ee1-199b-42c8-8011-0ad1c2ecfaab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:43:29.228920 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.228769    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-dshm" (OuterVolumeSpecName: "dshm") pod "082d8ee1-199b-42c8-8011-0ad1c2ecfaab" (UID: "082d8ee1-199b-42c8-8011-0ad1c2ecfaab"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:29.229037 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.228895    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-kube-api-access-dzdz8" (OuterVolumeSpecName: "kube-api-access-dzdz8") pod "082d8ee1-199b-42c8-8011-0ad1c2ecfaab" (UID: "082d8ee1-199b-42c8-8011-0ad1c2ecfaab"). InnerVolumeSpecName "kube-api-access-dzdz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:43:29.327937 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.327885    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzdz8\" (UniqueName: \"kubernetes.io/projected/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-kube-api-access-dzdz8\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:43:29.327937 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.327938    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:43:29.328099 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.327953    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/082d8ee1-199b-42c8-8011-0ad1c2ecfaab-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:43:29.502348 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.502318    2580 generic.go:356] "Generic (PLEG): container finished" podID="082d8ee1-199b-42c8-8011-0ad1c2ecfaab" containerID="308816049d653e420d10fea66c238e8b302b7195b223c9efebb30ef24da23e54" exitCode=0
Sep 11 13:43:29.502526 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.502363    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"
Sep 11 13:43:29.502526 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.502410    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9" event={"ID":"082d8ee1-199b-42c8-8011-0ad1c2ecfaab","Type":"ContainerDied","Data":"308816049d653e420d10fea66c238e8b302b7195b223c9efebb30ef24da23e54"}
Sep 11 13:43:29.502526 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.502451    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9" event={"ID":"082d8ee1-199b-42c8-8011-0ad1c2ecfaab","Type":"ContainerDied","Data":"30f1791908549f45f7af1f9cea6eae473f7da39bb5ccc102fc354beec13e46f7"}
Sep 11 13:43:29.502526 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.502475    2580 scope.go:117] "RemoveContainer" containerID="308816049d653e420d10fea66c238e8b302b7195b223c9efebb30ef24da23e54"
Sep 11 13:43:29.511508 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.511479    2580 scope.go:117] "RemoveContainer" containerID="308816049d653e420d10fea66c238e8b302b7195b223c9efebb30ef24da23e54"
Sep 11 13:43:29.511815 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:43:29.511798    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"308816049d653e420d10fea66c238e8b302b7195b223c9efebb30ef24da23e54\": container with ID starting with 308816049d653e420d10fea66c238e8b302b7195b223c9efebb30ef24da23e54 not found: ID does not exist" containerID="308816049d653e420d10fea66c238e8b302b7195b223c9efebb30ef24da23e54"
Sep 11 13:43:29.511889 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.511826    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"308816049d653e420d10fea66c238e8b302b7195b223c9efebb30ef24da23e54"} err="failed to get container status \"308816049d653e420d10fea66c238e8b302b7195b223c9efebb30ef24da23e54\": rpc error: code = NotFound desc = could not find container \"308816049d653e420d10fea66c238e8b302b7195b223c9efebb30ef24da23e54\": container with ID starting with 308816049d653e420d10fea66c238e8b302b7195b223c9efebb30ef24da23e54 not found: ID does not exist"
Sep 11 13:43:29.544644 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.544607    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"]
Sep 11 13:43:29.553469 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:29.553435    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-b65337/test-model-b65337-kserve-86bb57bdf-wr6g9"]
Sep 11 13:43:31.315974 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:31.315945    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="082d8ee1-199b-42c8-8011-0ad1c2ecfaab" path="/var/lib/kubelet/pods/082d8ee1-199b-42c8-8011-0ad1c2ecfaab/volumes"
Sep 11 13:43:42.357894 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.357847    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq" podUID="1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f" containerName="main" containerID="cri-o://a0a5421004ac491197b9ad039117042bcdeb07b7dff7d65cc2ae539dd54f9a5b" gracePeriod=45
Sep 11 13:43:42.582296 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.582265    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f" containerID="a0a5421004ac491197b9ad039117042bcdeb07b7dff7d65cc2ae539dd54f9a5b" exitCode=0
Sep 11 13:43:42.582453 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.582334    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq" event={"ID":"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f","Type":"ContainerDied","Data":"a0a5421004ac491197b9ad039117042bcdeb07b7dff7d65cc2ae539dd54f9a5b"}
Sep 11 13:43:42.613534 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.613482    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:43:42.755198 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.755162    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-tmp-dir\") pod \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") "
Sep 11 13:43:42.755198 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.755212    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftxvx\" (UniqueName: \"kubernetes.io/projected/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-kube-api-access-ftxvx\") pod \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") "
Sep 11 13:43:42.755450 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.755257    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-tls-certs\") pod \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") "
Sep 11 13:43:42.755450 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.755286    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-model-cache\") pod \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") "
Sep 11 13:43:42.755572 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.755453    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-dshm\") pod \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") "
Sep 11 13:43:42.755572 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.755504    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-home\") pod \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\" (UID: \"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f\") "
Sep 11 13:43:42.755572 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.755516    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f" (UID: "1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:42.755572 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.755540    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-model-cache" (OuterVolumeSpecName: "model-cache") pod "1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f" (UID: "1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:42.755794 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.755769    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-home" (OuterVolumeSpecName: "home") pod "1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f" (UID: "1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:42.755935 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.755918    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:43:42.756011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.755937    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:43:42.756011 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.755945    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:43:42.757579 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.757552    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-dshm" (OuterVolumeSpecName: "dshm") pod "1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f" (UID: "1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:42.757675 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.757595    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f" (UID: "1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:43:42.757675 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.757665    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-kube-api-access-ftxvx" (OuterVolumeSpecName: "kube-api-access-ftxvx") pod "1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f" (UID: "1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f"). InnerVolumeSpecName "kube-api-access-ftxvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:43:42.856821 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.856783    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftxvx\" (UniqueName: \"kubernetes.io/projected/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-kube-api-access-ftxvx\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:43:42.856821 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.856817    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:43:42.856821 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:42.856827    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:43:43.589346 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:43.589261    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"
Sep 11 13:43:43.589346 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:43.589310    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq" event={"ID":"1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f","Type":"ContainerDied","Data":"df69e23dee45c0689de50f31f22277679791298480f5970ce46459fcae9c328d"}
Sep 11 13:43:43.589816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:43.589362    2580 scope.go:117] "RemoveContainer" containerID="a0a5421004ac491197b9ad039117042bcdeb07b7dff7d65cc2ae539dd54f9a5b"
Sep 11 13:43:43.607790 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:43.607765    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"]
Sep 11 13:43:43.611662 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:43.611640    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-a649d3/test-model-a649d3-kserve-764787995-mfzhq"]
Sep 11 13:43:45.316197 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:45.316164    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f" path="/var/lib/kubelet/pods/1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f/volumes"
Sep 11 13:43:49.732280 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:49.732246    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a649d3-67884"]
Sep 11 13:43:49.732684 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:49.732671    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="082d8ee1-199b-42c8-8011-0ad1c2ecfaab" containerName="main"
Sep 11 13:43:49.732732 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:49.732685    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="082d8ee1-199b-42c8-8011-0ad1c2ecfaab" containerName="main"
Sep 11 13:43:49.732732 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:49.732695    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f" containerName="main"
Sep 11 13:43:49.732732 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:49.732700    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f" containerName="main"
Sep 11 13:43:49.732833 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:49.732772    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a967bf2-a01d-4e3c-8b0d-cd9ab3c4428f" containerName="main"
Sep 11 13:43:49.732833 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:49.732781    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="082d8ee1-199b-42c8-8011-0ad1c2ecfaab" containerName="main"
Sep 11 13:43:49.737248 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:49.737224    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a649d3-67884"
Sep 11 13:43:49.740209 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:49.740182    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:43:49.740349 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:49.740186    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:43:49.745998 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:49.745615    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a649d3-67884"]
Sep 11 13:43:49.824442 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:49.824401    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/072862fb-aa44-4dc0-a309-6dbdef473d9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a649d3-67884\" (UID: \"072862fb-aa44-4dc0-a309-6dbdef473d9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a649d3-67884"
Sep 11 13:43:49.925291 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:49.925254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/072862fb-aa44-4dc0-a309-6dbdef473d9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a649d3-67884\" (UID: \"072862fb-aa44-4dc0-a309-6dbdef473d9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a649d3-67884"
Sep 11 13:43:49.925890 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:49.925870    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/072862fb-aa44-4dc0-a309-6dbdef473d9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a649d3-67884\" (UID: \"072862fb-aa44-4dc0-a309-6dbdef473d9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a649d3-67884"
Sep 11 13:43:50.051109 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:50.051000    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a649d3-67884"
Sep 11 13:43:50.340735 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:50.340704    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a649d3-67884"]
Sep 11 13:43:50.341851 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:43:50.341824    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod072862fb_aa44_4dc0_a309_6dbdef473d9b.slice/crio-397c3f9a48b7e7cfb8c89edb95665cd940e867ce7c69bfc0938c4bae246649f0 WatchSource:0}: Error finding container 397c3f9a48b7e7cfb8c89edb95665cd940e867ce7c69bfc0938c4bae246649f0: Status 404 returned error can't find the container with id 397c3f9a48b7e7cfb8c89edb95665cd940e867ce7c69bfc0938c4bae246649f0
Sep 11 13:43:50.633795 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:50.633756    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a649d3-67884" event={"ID":"072862fb-aa44-4dc0-a309-6dbdef473d9b","Type":"ContainerStarted","Data":"397c3f9a48b7e7cfb8c89edb95665cd940e867ce7c69bfc0938c4bae246649f0"}
Sep 11 13:43:51.640927 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:51.640872    2580 generic.go:356] "Generic (PLEG): container finished" podID="072862fb-aa44-4dc0-a309-6dbdef473d9b" containerID="cccc576c552c307a081d5d527c87adea837edd6f41d45c4fc4ff14307844a65b" exitCode=0
Sep 11 13:43:51.641364 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:51.640960    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a649d3-67884" event={"ID":"072862fb-aa44-4dc0-a309-6dbdef473d9b","Type":"ContainerDied","Data":"cccc576c552c307a081d5d527c87adea837edd6f41d45c4fc4ff14307844a65b"}
Sep 11 13:43:52.783138 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:52.783111    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a649d3-67884"
Sep 11 13:43:52.956245 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:52.956155    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/072862fb-aa44-4dc0-a309-6dbdef473d9b-maas-api-service-ca\") pod \"072862fb-aa44-4dc0-a309-6dbdef473d9b\" (UID: \"072862fb-aa44-4dc0-a309-6dbdef473d9b\") "
Sep 11 13:43:52.956495 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:52.956471    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/072862fb-aa44-4dc0-a309-6dbdef473d9b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "072862fb-aa44-4dc0-a309-6dbdef473d9b" (UID: "072862fb-aa44-4dc0-a309-6dbdef473d9b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:43:53.056891 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:53.056855    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/072862fb-aa44-4dc0-a309-6dbdef473d9b-maas-api-service-ca\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:43:53.654220 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:53.654130    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a649d3-67884"
Sep 11 13:43:53.654220 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:53.654156    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a649d3-67884" event={"ID":"072862fb-aa44-4dc0-a309-6dbdef473d9b","Type":"ContainerDied","Data":"397c3f9a48b7e7cfb8c89edb95665cd940e867ce7c69bfc0938c4bae246649f0"}
Sep 11 13:43:53.654220 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:43:53.654209    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="397c3f9a48b7e7cfb8c89edb95665cd940e867ce7c69bfc0938c4bae246649f0"
Sep 11 13:44:00.766379 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:00.766345    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"]
Sep 11 13:44:00.766715 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:00.766542    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97" podUID="64567a04-9955-485b-a593-77e9733d51c7" containerName="payload-pre-processing" containerID="cri-o://c9a6522042cfea12e7edea4c85aaec08c0081562a4eddd2a384faa1c4d0dd905" gracePeriod=30
Sep 11 13:44:01.025576 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.025551    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"
Sep 11 13:44:01.127531 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.127436    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64567a04-9955-485b-a593-77e9733d51c7-plugins-config-volume\") pod \"64567a04-9955-485b-a593-77e9733d51c7\" (UID: \"64567a04-9955-485b-a593-77e9733d51c7\") "
Sep 11 13:44:01.127722 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.127619    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqb5q\" (UniqueName: \"kubernetes.io/projected/64567a04-9955-485b-a593-77e9733d51c7-kube-api-access-jqb5q\") pod \"64567a04-9955-485b-a593-77e9733d51c7\" (UID: \"64567a04-9955-485b-a593-77e9733d51c7\") "
Sep 11 13:44:01.127795 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.127736    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64567a04-9955-485b-a593-77e9733d51c7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "64567a04-9955-485b-a593-77e9733d51c7" (UID: "64567a04-9955-485b-a593-77e9733d51c7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:44:01.128020 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.128001    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64567a04-9955-485b-a593-77e9733d51c7-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:44:01.129696 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.129670    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64567a04-9955-485b-a593-77e9733d51c7-kube-api-access-jqb5q" (OuterVolumeSpecName: "kube-api-access-jqb5q") pod "64567a04-9955-485b-a593-77e9733d51c7" (UID: "64567a04-9955-485b-a593-77e9733d51c7"). InnerVolumeSpecName "kube-api-access-jqb5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:44:01.229346 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.229308    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqb5q\" (UniqueName: \"kubernetes.io/projected/64567a04-9955-485b-a593-77e9733d51c7-kube-api-access-jqb5q\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:44:01.241396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.241360    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a649d3-67884"]
Sep 11 13:44:01.247072 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.247043    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a649d3-67884"]
Sep 11 13:44:01.317235 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.317200    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="072862fb-aa44-4dc0-a309-6dbdef473d9b" path="/var/lib/kubelet/pods/072862fb-aa44-4dc0-a309-6dbdef473d9b/volumes"
Sep 11 13:44:01.702960 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.702841    2580 generic.go:356] "Generic (PLEG): container finished" podID="64567a04-9955-485b-a593-77e9733d51c7" containerID="c9a6522042cfea12e7edea4c85aaec08c0081562a4eddd2a384faa1c4d0dd905" exitCode=0
Sep 11 13:44:01.702960 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.702884    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"
Sep 11 13:44:01.702960 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.702925    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97" event={"ID":"64567a04-9955-485b-a593-77e9733d51c7","Type":"ContainerDied","Data":"c9a6522042cfea12e7edea4c85aaec08c0081562a4eddd2a384faa1c4d0dd905"}
Sep 11 13:44:01.703223 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.702987    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97" event={"ID":"64567a04-9955-485b-a593-77e9733d51c7","Type":"ContainerDied","Data":"7b8efca744c67ed3cbfc420585bce2fcf6fe227863f46af21e0646d0975830a2"}
Sep 11 13:44:01.703223 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.703017    2580 scope.go:117] "RemoveContainer" containerID="c9a6522042cfea12e7edea4c85aaec08c0081562a4eddd2a384faa1c4d0dd905"
Sep 11 13:44:01.712057 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.712038    2580 scope.go:117] "RemoveContainer" containerID="c9a6522042cfea12e7edea4c85aaec08c0081562a4eddd2a384faa1c4d0dd905"
Sep 11 13:44:01.712355 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:44:01.712330    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9a6522042cfea12e7edea4c85aaec08c0081562a4eddd2a384faa1c4d0dd905\": container with ID starting with c9a6522042cfea12e7edea4c85aaec08c0081562a4eddd2a384faa1c4d0dd905 not found: ID does not exist" containerID="c9a6522042cfea12e7edea4c85aaec08c0081562a4eddd2a384faa1c4d0dd905"
Sep 11 13:44:01.712450 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.712361    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9a6522042cfea12e7edea4c85aaec08c0081562a4eddd2a384faa1c4d0dd905"} err="failed to get container status \"c9a6522042cfea12e7edea4c85aaec08c0081562a4eddd2a384faa1c4d0dd905\": rpc error: code = NotFound desc = could not find container \"c9a6522042cfea12e7edea4c85aaec08c0081562a4eddd2a384faa1c4d0dd905\": container with ID starting with c9a6522042cfea12e7edea4c85aaec08c0081562a4eddd2a384faa1c4d0dd905 not found: ID does not exist"
Sep 11 13:44:01.739785 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.739754    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"]
Sep 11 13:44:01.746229 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:01.746204    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a649d3-656b8dfbbb-g6j97"]
Sep 11 13:44:02.546407 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:02.546371    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:44:03.316690 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:03.316657    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64567a04-9955-485b-a593-77e9733d51c7" path="/var/lib/kubelet/pods/64567a04-9955-485b-a593-77e9733d51c7/volumes"
Sep 11 13:44:21.141238 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.141202    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"]
Sep 11 13:44:21.141697 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.141630    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="072862fb-aa44-4dc0-a309-6dbdef473d9b" containerName="revoke-keys"
Sep 11 13:44:21.141697 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.141645    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="072862fb-aa44-4dc0-a309-6dbdef473d9b" containerName="revoke-keys"
Sep 11 13:44:21.141697 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.141658    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64567a04-9955-485b-a593-77e9733d51c7" containerName="payload-pre-processing"
Sep 11 13:44:21.141697 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.141664    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="64567a04-9955-485b-a593-77e9733d51c7" containerName="payload-pre-processing"
Sep 11 13:44:21.141835 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.141748    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="072862fb-aa44-4dc0-a309-6dbdef473d9b" containerName="revoke-keys"
Sep 11 13:44:21.141835 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.141760    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="64567a04-9955-485b-a593-77e9733d51c7" containerName="payload-pre-processing"
Sep 11 13:44:21.144771 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.144752    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"
Sep 11 13:44:21.148448 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.148419    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-0317c3\""
Sep 11 13:44:21.155414 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.155385    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"]
Sep 11 13:44:21.224463 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.224430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1da49c8-c4fe-411b-89e5-e08bbcfea7db-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz\" (UID: \"e1da49c8-c4fe-411b-89e5-e08bbcfea7db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"
Sep 11 13:44:21.224626 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.224523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzcwr\" (UniqueName: \"kubernetes.io/projected/e1da49c8-c4fe-411b-89e5-e08bbcfea7db-kube-api-access-qzcwr\") pod \"payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz\" (UID: \"e1da49c8-c4fe-411b-89e5-e08bbcfea7db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"
Sep 11 13:44:21.325548 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.325510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzcwr\" (UniqueName: \"kubernetes.io/projected/e1da49c8-c4fe-411b-89e5-e08bbcfea7db-kube-api-access-qzcwr\") pod \"payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz\" (UID: \"e1da49c8-c4fe-411b-89e5-e08bbcfea7db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"
Sep 11 13:44:21.325729 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.325624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1da49c8-c4fe-411b-89e5-e08bbcfea7db-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz\" (UID: \"e1da49c8-c4fe-411b-89e5-e08bbcfea7db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"
Sep 11 13:44:21.329439 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.329418    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-0317c3\""
Sep 11 13:44:21.336368 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.336348    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1da49c8-c4fe-411b-89e5-e08bbcfea7db-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz\" (UID: \"e1da49c8-c4fe-411b-89e5-e08bbcfea7db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"
Sep 11 13:44:21.337702 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.337679    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzcwr\" (UniqueName: \"kubernetes.io/projected/e1da49c8-c4fe-411b-89e5-e08bbcfea7db-kube-api-access-qzcwr\") pod \"payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz\" (UID: \"e1da49c8-c4fe-411b-89e5-e08bbcfea7db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"
Sep 11 13:44:21.456057 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.455961    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"
Sep 11 13:44:21.497963 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.497930    2580 scope.go:117] "RemoveContainer" containerID="93336ec45608f3d79c48bb4028e57789b1c5191252d04842e0908df53c26ec8e"
Sep 11 13:44:21.738037 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.737952    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"]
Sep 11 13:44:21.740563 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:44:21.740535    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1da49c8_c4fe_411b_89e5_e08bbcfea7db.slice/crio-f454c7ffc3552239ed66d73ec3f2dd1f03b1a6fbd781f32dca8b61126303da58 WatchSource:0}: Error finding container f454c7ffc3552239ed66d73ec3f2dd1f03b1a6fbd781f32dca8b61126303da58: Status 404 returned error can't find the container with id f454c7ffc3552239ed66d73ec3f2dd1f03b1a6fbd781f32dca8b61126303da58
Sep 11 13:44:21.825869 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.825833    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz" event={"ID":"e1da49c8-c4fe-411b-89e5-e08bbcfea7db","Type":"ContainerStarted","Data":"2b2de0799a80548fd37e1a84f856424f8cbd7fe1a3fc7e1e50e457cf950ec2c9"}
Sep 11 13:44:21.826036 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.825891    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz" event={"ID":"e1da49c8-c4fe-411b-89e5-e08bbcfea7db","Type":"ContainerStarted","Data":"f454c7ffc3552239ed66d73ec3f2dd1f03b1a6fbd781f32dca8b61126303da58"}
Sep 11 13:44:21.826036 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.826002    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"
Sep 11 13:44:21.854630 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:21.854574    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz" podStartSLOduration=0.854557637 podStartE2EDuration="854.557637ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:44:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:44:21.851798663 +0000 UTC m=+1381.070336156" watchObservedRunningTime="2026-09-11 13:44:21.854557637 +0000 UTC m=+1381.073095128"
Sep 11 13:44:24.054977 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.054943    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"]
Sep 11 13:44:24.058447 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.058430    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.061059 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.061020    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c0f589fb\"/\"default-dockercfg-97xpt\""
Sep 11 13:44:24.061368 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.061349    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c0f589fb\"/\"kube-root-ca.crt\""
Sep 11 13:44:24.062200 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.062182    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c0f589fb\"/\"openshift-service-ca.crt\""
Sep 11 13:44:24.062265 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.062247    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c0f589fb\"/\"e2e-lifecycle-model-c0f589fb-kserve-self-signed-certs\""
Sep 11 13:44:24.069062 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.069040    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"]
Sep 11 13:44:24.151625 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.151592    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-model-cache\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.151799 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.151637    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-tls-certs\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.151799 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.151660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-tmp-dir\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.151799 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.151732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-home\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.151799 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.151770    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-dshm\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.151972 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.151800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l227\" (UniqueName: \"kubernetes.io/projected/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-kube-api-access-2l227\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.252632 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.252591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-tls-certs\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.252632 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.252632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-tmp-dir\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.252844 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.252657    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-home\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.252844 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.252700    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-dshm\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.252844 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.252742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l227\" (UniqueName: \"kubernetes.io/projected/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-kube-api-access-2l227\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.253030 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.252853    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-model-cache\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.253131 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.253108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-tmp-dir\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.253206 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.253180    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-home\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.253324 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.253302    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-model-cache\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.255014 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.254991    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-dshm\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.255338 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.255319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-tls-certs\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.262498 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.262473    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l227\" (UniqueName: \"kubernetes.io/projected/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-kube-api-access-2l227\") pod \"e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") " pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.367877 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.367841    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:24.657403 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.657377    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"]
Sep 11 13:44:24.660061 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:44:24.660030    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda96ccaa4_844a_4b38_9bd8_0c11bbcebb5d.slice/crio-7cc5588fde2fa9ea40755ac3a024df132f9fb42588cb2770c45e967cf5186102 WatchSource:0}: Error finding container 7cc5588fde2fa9ea40755ac3a024df132f9fb42588cb2770c45e967cf5186102: Status 404 returned error can't find the container with id 7cc5588fde2fa9ea40755ac3a024df132f9fb42588cb2770c45e967cf5186102
Sep 11 13:44:24.844590 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.844553    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z" event={"ID":"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d","Type":"ContainerStarted","Data":"50206806b8ffa3724a7fd8096d29c18f8032209922f2e1d5a9a21852d9daaee2"}
Sep 11 13:44:24.844590 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.844598    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z" event={"ID":"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d","Type":"ContainerStarted","Data":"7cc5588fde2fa9ea40755ac3a024df132f9fb42588cb2770c45e967cf5186102"}
Sep 11 13:44:24.870969 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:24.870881    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z" podStartSLOduration=0.870861181 podStartE2EDuration="870.861181ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:44:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:44:24.869644127 +0000 UTC m=+1384.088181618" watchObservedRunningTime="2026-09-11 13:44:24.870861181 +0000 UTC m=+1384.089398678"
Sep 11 13:44:32.834077 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:32.834044    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"
Sep 11 13:44:33.854418 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:33.854383    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:44:34.368143 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:34.368105    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:34.368452 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:34.368419    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:34.381384 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:34.381351    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:34.914064 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:34.914034    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:44:52.865764 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:52.865730    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"]
Sep 11 13:44:52.869298 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:52.869281    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"
Sep 11 13:44:52.872119 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:52.872091    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-9edb53-dockercfg-lcpqg\""
Sep 11 13:44:52.872216 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:52.872165    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-9edb53\""
Sep 11 13:44:52.883031 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:52.883011    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"]
Sep 11 13:44:52.937981 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:52.937942    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6czst\" (UniqueName: \"kubernetes.io/projected/014bbf30-59b1-45f2-a2dc-1449b1eafb1f-kube-api-access-6czst\") pod \"payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp\" (UID: \"014bbf30-59b1-45f2-a2dc-1449b1eafb1f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"
Sep 11 13:44:52.938156 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:52.937993    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/014bbf30-59b1-45f2-a2dc-1449b1eafb1f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp\" (UID: \"014bbf30-59b1-45f2-a2dc-1449b1eafb1f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"
Sep 11 13:44:53.039292 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:53.039253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6czst\" (UniqueName: \"kubernetes.io/projected/014bbf30-59b1-45f2-a2dc-1449b1eafb1f-kube-api-access-6czst\") pod \"payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp\" (UID: \"014bbf30-59b1-45f2-a2dc-1449b1eafb1f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"
Sep 11 13:44:53.039292 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:53.039291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/014bbf30-59b1-45f2-a2dc-1449b1eafb1f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp\" (UID: \"014bbf30-59b1-45f2-a2dc-1449b1eafb1f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"
Sep 11 13:44:53.040064 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:53.040044    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/014bbf30-59b1-45f2-a2dc-1449b1eafb1f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp\" (UID: \"014bbf30-59b1-45f2-a2dc-1449b1eafb1f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"
Sep 11 13:44:53.048538 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:53.048517    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6czst\" (UniqueName: \"kubernetes.io/projected/014bbf30-59b1-45f2-a2dc-1449b1eafb1f-kube-api-access-6czst\") pod \"payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp\" (UID: \"014bbf30-59b1-45f2-a2dc-1449b1eafb1f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"
Sep 11 13:44:53.180403 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:53.180317    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"
Sep 11 13:44:53.466715 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:53.466673    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"]
Sep 11 13:44:53.466945 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:44:53.466922    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod014bbf30_59b1_45f2_a2dc_1449b1eafb1f.slice/crio-dcfa69f5182d8ea2770bfe0ff05d7a970740f3d98118ffabc4b9010ad8c4f973 WatchSource:0}: Error finding container dcfa69f5182d8ea2770bfe0ff05d7a970740f3d98118ffabc4b9010ad8c4f973: Status 404 returned error can't find the container with id dcfa69f5182d8ea2770bfe0ff05d7a970740f3d98118ffabc4b9010ad8c4f973
Sep 11 13:44:54.015699 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:54.015662    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp" event={"ID":"014bbf30-59b1-45f2-a2dc-1449b1eafb1f","Type":"ContainerStarted","Data":"9f987ed54d471ce6243129d8a1b766c5494c3b08a93aea795aebc12fee988a42"}
Sep 11 13:44:54.016089 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:54.015721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp" event={"ID":"014bbf30-59b1-45f2-a2dc-1449b1eafb1f","Type":"ContainerStarted","Data":"dcfa69f5182d8ea2770bfe0ff05d7a970740f3d98118ffabc4b9010ad8c4f973"}
Sep 11 13:44:54.016089 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:54.015858    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"
Sep 11 13:44:54.037605 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:44:54.037549    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp" podStartSLOduration=2.037533556 podStartE2EDuration="2.037533556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:44:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:44:54.036265537 +0000 UTC m=+1413.254803030" watchObservedRunningTime="2026-09-11 13:44:54.037533556 +0000 UTC m=+1413.256071048"
Sep 11 13:45:00.210998 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.210432    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-470eeb-29818905-lpk2d"]
Sep 11 13:45:00.217713 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.217678    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-470eeb-29818905-lpk2d"
Sep 11 13:45:00.222513 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.222486    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:45:00.232072 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.231053    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-470eeb-29818905-lpk2d"
Sep 11 13:45:00.240386 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.240345    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-470eeb-29818905-lpk2d"]
Sep 11 13:45:00.243099 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.243075    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818905-q2bv2"]
Sep 11 13:45:00.246861 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.246837    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818905-q2bv2"
Sep 11 13:45:00.257388 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.257353    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818905-q2bv2"]
Sep 11 13:45:00.273431 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.273402    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818905-q2bv2"
Sep 11 13:45:00.283077 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.281562    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0317c3-29818905-frz76"]
Sep 11 13:45:00.286133 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.286104    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0317c3-29818905-frz76"
Sep 11 13:45:00.290242 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.290212    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-e52aaa-29818905-8v4lm"]
Sep 11 13:45:00.300371 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.299195    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-e52aaa-29818905-8v4lm"
Sep 11 13:45:00.307552 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.307521    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0317c3-29818905-frz76"]
Sep 11 13:45:00.326613 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.326564    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-e52aaa-29818905-8v4lm"]
Sep 11 13:45:00.336665 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.335544    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0317c3-29818905-frz76"
Sep 11 13:45:00.344401 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.344297    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-e52aaa-29818905-8v4lm"
Sep 11 13:45:00.532167 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.532129    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-470eeb-29818905-lpk2d"]
Sep 11 13:45:00.534862 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:45:00.534832    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod893c3866_dd59_48d6_8b25_4d1c832bea7c.slice/crio-692218e78983ba95ebf286e131f671e7c292090f026504cd40718d828a8b63ec WatchSource:0}: Error finding container 692218e78983ba95ebf286e131f671e7c292090f026504cd40718d828a8b63ec: Status 404 returned error can't find the container with id 692218e78983ba95ebf286e131f671e7c292090f026504cd40718d828a8b63ec
Sep 11 13:45:00.583972 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.583946    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818905-q2bv2"]
Sep 11 13:45:00.586226 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:45:00.586199    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5fbe1c1_68f4_436b_bf35_47b7694d82b6.slice/crio-c5c8242b6ee1b1859a165679283111c911599621c310960bc1a884c9accb12dc WatchSource:0}: Error finding container c5c8242b6ee1b1859a165679283111c911599621c310960bc1a884c9accb12dc: Status 404 returned error can't find the container with id c5c8242b6ee1b1859a165679283111c911599621c310960bc1a884c9accb12dc
Sep 11 13:45:00.633260 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.633232    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0317c3-29818905-frz76"]
Sep 11 13:45:00.633501 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:45:00.633482    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81916f38_81a9_4b6a_b13f_e0d19557883d.slice/crio-31afe11270af2b7ee6462b661fe05d3ef3020ebec214b2f9f0cbef1092d2caf5 WatchSource:0}: Error finding container 31afe11270af2b7ee6462b661fe05d3ef3020ebec214b2f9f0cbef1092d2caf5: Status 404 returned error can't find the container with id 31afe11270af2b7ee6462b661fe05d3ef3020ebec214b2f9f0cbef1092d2caf5
Sep 11 13:45:00.647374 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:00.647349    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-e52aaa-29818905-8v4lm"]
Sep 11 13:45:00.648569 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:45:00.648546    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31489940_f821_4263_a128_fbe7aeeeccb4.slice/crio-2a8156ef20cfb07df66e247c7672b43b71c64c6e0b7aab2cc4669e9519f1b3ef WatchSource:0}: Error finding container 2a8156ef20cfb07df66e247c7672b43b71c64c6e0b7aab2cc4669e9519f1b3ef: Status 404 returned error can't find the container with id 2a8156ef20cfb07df66e247c7672b43b71c64c6e0b7aab2cc4669e9519f1b3ef
Sep 11 13:45:01.058126 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:01.058038    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-e52aaa-29818905-8v4lm" event={"ID":"31489940-f821-4263-a128-fbe7aeeeccb4","Type":"ContainerStarted","Data":"c3e6104433e578eb981f158eec6cbe6006f75ada8fd859c04e4050347dc1d35f"}
Sep 11 13:45:01.058126 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:01.058099    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-e52aaa-29818905-8v4lm" event={"ID":"31489940-f821-4263-a128-fbe7aeeeccb4","Type":"ContainerStarted","Data":"2a8156ef20cfb07df66e247c7672b43b71c64c6e0b7aab2cc4669e9519f1b3ef"}
Sep 11 13:45:01.059810 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:01.059778    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0317c3-29818905-frz76" event={"ID":"81916f38-81a9-4b6a-b13f-e0d19557883d","Type":"ContainerStarted","Data":"b08f561e822684091ff7d5594453b5d1ecdad4e66686009dd137ab5023c8bc73"}
Sep 11 13:45:01.059948 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:01.059823    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0317c3-29818905-frz76" event={"ID":"81916f38-81a9-4b6a-b13f-e0d19557883d","Type":"ContainerStarted","Data":"31afe11270af2b7ee6462b661fe05d3ef3020ebec214b2f9f0cbef1092d2caf5"}
Sep 11 13:45:01.061311 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:01.061283    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818905-q2bv2" event={"ID":"e5fbe1c1-68f4-436b-bf35-47b7694d82b6","Type":"ContainerStarted","Data":"ee113ac355f8bbbc9feebca4a3a75bf70c64f96d844379834bf6bd9a882f2a3e"}
Sep 11 13:45:01.061484 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:01.061332    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818905-q2bv2" event={"ID":"e5fbe1c1-68f4-436b-bf35-47b7694d82b6","Type":"ContainerStarted","Data":"c5c8242b6ee1b1859a165679283111c911599621c310960bc1a884c9accb12dc"}
Sep 11 13:45:01.062781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:01.062761    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-470eeb-29818905-lpk2d" event={"ID":"893c3866-dd59-48d6-8b25-4d1c832bea7c","Type":"ContainerStarted","Data":"6b853b6e724046730c2342b66bbcf429088f38d0345cf30e5e87a39ebcf4e6c3"}
Sep 11 13:45:01.062885 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:01.062797    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-470eeb-29818905-lpk2d" event={"ID":"893c3866-dd59-48d6-8b25-4d1c832bea7c","Type":"ContainerStarted","Data":"692218e78983ba95ebf286e131f671e7c292090f026504cd40718d828a8b63ec"}
Sep 11 13:45:01.084363 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:01.084304    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-e52aaa-29818905-8v4lm" podStartSLOduration=1.084285566 podStartE2EDuration="1.084285566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:45:01.082847079 +0000 UTC m=+1420.301384572" watchObservedRunningTime="2026-09-11 13:45:01.084285566 +0000 UTC m=+1420.302823059"
Sep 11 13:45:01.103863 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:01.103814    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-470eeb-29818905-lpk2d" podStartSLOduration=1.103799454 podStartE2EDuration="1.103799454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:45:01.101007384 +0000 UTC m=+1420.319544879" watchObservedRunningTime="2026-09-11 13:45:01.103799454 +0000 UTC m=+1420.322336960"
Sep 11 13:45:01.121496 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:01.121356    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0317c3-29818905-frz76" podStartSLOduration=1.121337259 podStartE2EDuration="1.121337259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:45:01.12056968 +0000 UTC m=+1420.339107173" watchObservedRunningTime="2026-09-11 13:45:01.121337259 +0000 UTC m=+1420.339874751"
Sep 11 13:45:01.145277 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:01.145227    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818905-q2bv2" podStartSLOduration=1.145211226 podStartE2EDuration="1.145211226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:45:01.145063479 +0000 UTC m=+1420.363600971" watchObservedRunningTime="2026-09-11 13:45:01.145211226 +0000 UTC m=+1420.363748717"
Sep 11 13:45:02.070602 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:02.070566    2580 generic.go:356] "Generic (PLEG): container finished" podID="31489940-f821-4263-a128-fbe7aeeeccb4" containerID="c3e6104433e578eb981f158eec6cbe6006f75ada8fd859c04e4050347dc1d35f" exitCode=0
Sep 11 13:45:02.071027 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:02.070651    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-e52aaa-29818905-8v4lm" event={"ID":"31489940-f821-4263-a128-fbe7aeeeccb4","Type":"ContainerDied","Data":"c3e6104433e578eb981f158eec6cbe6006f75ada8fd859c04e4050347dc1d35f"}
Sep 11 13:45:02.072219 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:02.072194    2580 generic.go:356] "Generic (PLEG): container finished" podID="81916f38-81a9-4b6a-b13f-e0d19557883d" containerID="b08f561e822684091ff7d5594453b5d1ecdad4e66686009dd137ab5023c8bc73" exitCode=0
Sep 11 13:45:02.072300 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:02.072280    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0317c3-29818905-frz76" event={"ID":"81916f38-81a9-4b6a-b13f-e0d19557883d","Type":"ContainerDied","Data":"b08f561e822684091ff7d5594453b5d1ecdad4e66686009dd137ab5023c8bc73"}
Sep 11 13:45:02.073756 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:02.073734    2580 generic.go:356] "Generic (PLEG): container finished" podID="e5fbe1c1-68f4-436b-bf35-47b7694d82b6" containerID="ee113ac355f8bbbc9feebca4a3a75bf70c64f96d844379834bf6bd9a882f2a3e" exitCode=0
Sep 11 13:45:02.073825 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:02.073806    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818905-q2bv2" event={"ID":"e5fbe1c1-68f4-436b-bf35-47b7694d82b6","Type":"ContainerDied","Data":"ee113ac355f8bbbc9feebca4a3a75bf70c64f96d844379834bf6bd9a882f2a3e"}
Sep 11 13:45:02.075243 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:02.075220    2580 generic.go:356] "Generic (PLEG): container finished" podID="893c3866-dd59-48d6-8b25-4d1c832bea7c" containerID="6b853b6e724046730c2342b66bbcf429088f38d0345cf30e5e87a39ebcf4e6c3" exitCode=0
Sep 11 13:45:02.075334 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:02.075246    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-470eeb-29818905-lpk2d" event={"ID":"893c3866-dd59-48d6-8b25-4d1c832bea7c","Type":"ContainerDied","Data":"6b853b6e724046730c2342b66bbcf429088f38d0345cf30e5e87a39ebcf4e6c3"}
Sep 11 13:45:02.691554 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:02.691522    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:45:03.251816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:03.251786    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-470eeb-29818905-lpk2d"
Sep 11 13:45:03.306918 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:03.306880    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818905-q2bv2"
Sep 11 13:45:03.310297 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:03.310275    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0317c3-29818905-frz76"
Sep 11 13:45:03.314528 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:03.314503    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-e52aaa-29818905-8v4lm"
Sep 11 13:45:04.093467 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.093431    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0317c3-29818905-frz76"
Sep 11 13:45:04.093772 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.093481    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0317c3-29818905-frz76" event={"ID":"81916f38-81a9-4b6a-b13f-e0d19557883d","Type":"ContainerDied","Data":"31afe11270af2b7ee6462b661fe05d3ef3020ebec214b2f9f0cbef1092d2caf5"}
Sep 11 13:45:04.093772 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.093537    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31afe11270af2b7ee6462b661fe05d3ef3020ebec214b2f9f0cbef1092d2caf5"
Sep 11 13:45:04.094983 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.094964    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818905-q2bv2"
Sep 11 13:45:04.095110 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.094997    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818905-q2bv2" event={"ID":"e5fbe1c1-68f4-436b-bf35-47b7694d82b6","Type":"ContainerDied","Data":"c5c8242b6ee1b1859a165679283111c911599621c310960bc1a884c9accb12dc"}
Sep 11 13:45:04.095110 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.095041    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5c8242b6ee1b1859a165679283111c911599621c310960bc1a884c9accb12dc"
Sep 11 13:45:04.096422 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.096403    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-470eeb-29818905-lpk2d"
Sep 11 13:45:04.096551 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.096400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-470eeb-29818905-lpk2d" event={"ID":"893c3866-dd59-48d6-8b25-4d1c832bea7c","Type":"ContainerDied","Data":"692218e78983ba95ebf286e131f671e7c292090f026504cd40718d828a8b63ec"}
Sep 11 13:45:04.096551 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.096517    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="692218e78983ba95ebf286e131f671e7c292090f026504cd40718d828a8b63ec"
Sep 11 13:45:04.097943 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.097897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-e52aaa-29818905-8v4lm" event={"ID":"31489940-f821-4263-a128-fbe7aeeeccb4","Type":"ContainerDied","Data":"2a8156ef20cfb07df66e247c7672b43b71c64c6e0b7aab2cc4669e9519f1b3ef"}
Sep 11 13:45:04.097943 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.097943    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a8156ef20cfb07df66e247c7672b43b71c64c6e0b7aab2cc4669e9519f1b3ef"
Sep 11 13:45:04.098121 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.097947    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-e52aaa-29818905-8v4lm"
Sep 11 13:45:04.797803 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.797769    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"]
Sep 11 13:45:04.798210 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.798197    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="893c3866-dd59-48d6-8b25-4d1c832bea7c" containerName="cleanup"
Sep 11 13:45:04.798210 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.798211    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="893c3866-dd59-48d6-8b25-4d1c832bea7c" containerName="cleanup"
Sep 11 13:45:04.798285 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.798223    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81916f38-81a9-4b6a-b13f-e0d19557883d" containerName="cleanup"
Sep 11 13:45:04.798285 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.798230    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="81916f38-81a9-4b6a-b13f-e0d19557883d" containerName="cleanup"
Sep 11 13:45:04.798285 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.798238    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5fbe1c1-68f4-436b-bf35-47b7694d82b6" containerName="cleanup"
Sep 11 13:45:04.798285 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.798243    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5fbe1c1-68f4-436b-bf35-47b7694d82b6" containerName="cleanup"
Sep 11 13:45:04.798285 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.798265    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31489940-f821-4263-a128-fbe7aeeeccb4" containerName="cleanup"
Sep 11 13:45:04.798285 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.798271    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="31489940-f821-4263-a128-fbe7aeeeccb4" containerName="cleanup"
Sep 11 13:45:04.798471 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.798336    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="893c3866-dd59-48d6-8b25-4d1c832bea7c" containerName="cleanup"
Sep 11 13:45:04.798471 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.798345    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5fbe1c1-68f4-436b-bf35-47b7694d82b6" containerName="cleanup"
Sep 11 13:45:04.798471 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.798369    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="81916f38-81a9-4b6a-b13f-e0d19557883d" containerName="cleanup"
Sep 11 13:45:04.798471 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.798375    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="31489940-f821-4263-a128-fbe7aeeeccb4" containerName="cleanup"
Sep 11 13:45:04.801088 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.801072    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:04.806078 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.806055    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-371c38\"/\"e2e-conflict-b-f8ce9a34-kserve-self-signed-certs\""
Sep 11 13:45:04.821251 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.821227    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"]
Sep 11 13:45:04.962723 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.962687    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-home\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:04.962899 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.962744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90689975-f2c9-4806-8ec5-32eb740022f8-tls-certs\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:04.962899 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.962800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-tmp-dir\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:04.962899 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.962867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-model-cache\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:04.963050 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.962945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-dshm\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:04.963050 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:04.962973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgfs5\" (UniqueName: \"kubernetes.io/projected/90689975-f2c9-4806-8ec5-32eb740022f8-kube-api-access-jgfs5\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:05.024544 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:05.024512    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"
Sep 11 13:45:05.064049 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:05.064021    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-dshm\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:05.064231 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:05.064072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgfs5\" (UniqueName: \"kubernetes.io/projected/90689975-f2c9-4806-8ec5-32eb740022f8-kube-api-access-jgfs5\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:05.064231 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:05.064133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-home\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:05.064231 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:05.064192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90689975-f2c9-4806-8ec5-32eb740022f8-tls-certs\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:05.064231 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:05.064222    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-tmp-dir\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:05.064441 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:05.064259    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-model-cache\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:05.064556 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:05.064530    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-home\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:05.064630 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:05.064615    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-tmp-dir\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:05.064703 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:05.064678    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-model-cache\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:05.066291 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:05.066263    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-dshm\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:05.066704 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:05.066684    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90689975-f2c9-4806-8ec5-32eb740022f8-tls-certs\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:05.073842 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:05.073815    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgfs5\" (UniqueName: \"kubernetes.io/projected/90689975-f2c9-4806-8ec5-32eb740022f8-kube-api-access-jgfs5\") pod \"e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") " pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:05.110714 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:05.110680    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:05.403420 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:45:05.403391    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90689975_f2c9_4806_8ec5_32eb740022f8.slice/crio-cf448fdab7b4ab4100c5a7352178006550292eb75c2ca9a0d539107d1e7fc285 WatchSource:0}: Error finding container cf448fdab7b4ab4100c5a7352178006550292eb75c2ca9a0d539107d1e7fc285: Status 404 returned error can't find the container with id cf448fdab7b4ab4100c5a7352178006550292eb75c2ca9a0d539107d1e7fc285
Sep 11 13:45:05.403729 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:05.403704    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"]
Sep 11 13:45:06.112586 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:06.112554    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc" event={"ID":"90689975-f2c9-4806-8ec5-32eb740022f8","Type":"ContainerStarted","Data":"0c0fc6bd96dd8e9e3ef7d8346029c7b4de48ece33a916cfc8aeab8697d0c4f23"}
Sep 11 13:45:06.112586 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:06.112592    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc" event={"ID":"90689975-f2c9-4806-8ec5-32eb740022f8","Type":"ContainerStarted","Data":"cf448fdab7b4ab4100c5a7352178006550292eb75c2ca9a0d539107d1e7fc285"}
Sep 11 13:45:06.139176 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:06.139128    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc" podStartSLOduration=2.139112914 podStartE2EDuration="2.139112914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:45:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:45:06.136148779 +0000 UTC m=+1425.354686270" watchObservedRunningTime="2026-09-11 13:45:06.139112914 +0000 UTC m=+1425.357650451"
Sep 11 13:45:15.111263 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:15.111152    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:15.111263 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:15.111223    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:15.123549 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:15.123519    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:15.185701 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:15.185674    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:45:26.523067 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.523031    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"]
Sep 11 13:45:26.527087 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.527059    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.541370 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.541346    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-082f62\"/\"distinct-2-w0-082f62-kserve-self-signed-certs\""
Sep 11 13:45:26.556794 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.556764    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"]
Sep 11 13:45:26.578506 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.578478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-tmp-dir\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.578506 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.578508    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-dshm\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.578678 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.578543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5gf8\" (UniqueName: \"kubernetes.io/projected/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-kube-api-access-s5gf8\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.578678 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.578599    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-home\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.578678 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.578624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-model-cache\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.578792 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.578683    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-tls-certs\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.679247 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.679215    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-tmp-dir\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.679247 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.679247    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-dshm\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.679458 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.679271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5gf8\" (UniqueName: \"kubernetes.io/projected/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-kube-api-access-s5gf8\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.679458 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.679311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-home\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.679458 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.679339    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-model-cache\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.679458 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.679412    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-tls-certs\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.679682 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.679657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-tmp-dir\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.679764 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.679735    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-home\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.679876 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.679768    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-model-cache\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.681607 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.681585    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-dshm\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.681877 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.681859    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-tls-certs\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.710435 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.710404    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5gf8\" (UniqueName: \"kubernetes.io/projected/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-kube-api-access-s5gf8\") pod \"distinct-2-w0-082f62-kserve-5665b9cf6-5brnq\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") " pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:26.838963 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:26.838926    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:27.122498 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:27.122474    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"]
Sep 11 13:45:27.123871 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:45:27.123846    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bb69ce4_5d77_4d0c_abfe_517da5af7bf5.slice/crio-6b9a5525555e1f7649a117bc357d335e641c11c1010af87223ad42084cc63871 WatchSource:0}: Error finding container 6b9a5525555e1f7649a117bc357d335e641c11c1010af87223ad42084cc63871: Status 404 returned error can't find the container with id 6b9a5525555e1f7649a117bc357d335e641c11c1010af87223ad42084cc63871
Sep 11 13:45:27.249970 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:27.249933    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq" event={"ID":"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5","Type":"ContainerStarted","Data":"f0bbe182d11fd402363556b2e6ad54e36dfd84e7f764e766ab4253ddd3aa0379"}
Sep 11 13:45:27.250158 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:27.249985    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq" event={"ID":"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5","Type":"ContainerStarted","Data":"6b9a5525555e1f7649a117bc357d335e641c11c1010af87223ad42084cc63871"}
Sep 11 13:45:27.271529 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:27.271481    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq" podStartSLOduration=1.271465482 podStartE2EDuration="1.271465482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:45:27.270383877 +0000 UTC m=+1446.488921371" watchObservedRunningTime="2026-09-11 13:45:27.271465482 +0000 UTC m=+1446.490003026"
Sep 11 13:45:36.839672 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:36.839634    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:36.840318 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:36.839705    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:36.852402 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:36.852379    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:37.344291 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:37.344258    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:45:39.676625 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:45:39.676587    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:46:10.912882 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:10.912834    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:46:16.325124 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:16.325086    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"]
Sep 11 13:46:16.325565 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:16.325275    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl" podUID="7fc6cfad-c77d-49cf-b6f1-fb4e4466a335" containerName="payload-pre-processing" containerID="cri-o://f400c0f1e521f41fe8f6b2e90bb321cc4d76185dcba60e8c79abca0fc810feca" gracePeriod=30
Sep 11 13:46:16.569879 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:16.569843    2580 generic.go:356] "Generic (PLEG): container finished" podID="7fc6cfad-c77d-49cf-b6f1-fb4e4466a335" containerID="f400c0f1e521f41fe8f6b2e90bb321cc4d76185dcba60e8c79abca0fc810feca" exitCode=0
Sep 11 13:46:16.570031 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:16.569935    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl" event={"ID":"7fc6cfad-c77d-49cf-b6f1-fb4e4466a335","Type":"ContainerDied","Data":"f400c0f1e521f41fe8f6b2e90bb321cc4d76185dcba60e8c79abca0fc810feca"}
Sep 11 13:46:16.570031 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:16.570016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl" event={"ID":"7fc6cfad-c77d-49cf-b6f1-fb4e4466a335","Type":"ContainerDied","Data":"ae4633f879263a3a1063f53046ce81bae113a0304996a5b61e968f434cebd52b"}
Sep 11 13:46:16.570109 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:16.570033    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae4633f879263a3a1063f53046ce81bae113a0304996a5b61e968f434cebd52b"
Sep 11 13:46:16.580559 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:16.580508    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"
Sep 11 13:46:16.669459 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:16.669428    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvzm5\" (UniqueName: \"kubernetes.io/projected/7fc6cfad-c77d-49cf-b6f1-fb4e4466a335-kube-api-access-lvzm5\") pod \"7fc6cfad-c77d-49cf-b6f1-fb4e4466a335\" (UID: \"7fc6cfad-c77d-49cf-b6f1-fb4e4466a335\") "
Sep 11 13:46:16.669623 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:16.669525    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fc6cfad-c77d-49cf-b6f1-fb4e4466a335-plugins-config-volume\") pod \"7fc6cfad-c77d-49cf-b6f1-fb4e4466a335\" (UID: \"7fc6cfad-c77d-49cf-b6f1-fb4e4466a335\") "
Sep 11 13:46:16.669961 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:16.669933    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fc6cfad-c77d-49cf-b6f1-fb4e4466a335-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7fc6cfad-c77d-49cf-b6f1-fb4e4466a335" (UID: "7fc6cfad-c77d-49cf-b6f1-fb4e4466a335"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:46:16.671696 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:16.671662    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fc6cfad-c77d-49cf-b6f1-fb4e4466a335-kube-api-access-lvzm5" (OuterVolumeSpecName: "kube-api-access-lvzm5") pod "7fc6cfad-c77d-49cf-b6f1-fb4e4466a335" (UID: "7fc6cfad-c77d-49cf-b6f1-fb4e4466a335"). InnerVolumeSpecName "kube-api-access-lvzm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:46:16.770603 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:16.770566    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvzm5\" (UniqueName: \"kubernetes.io/projected/7fc6cfad-c77d-49cf-b6f1-fb4e4466a335-kube-api-access-lvzm5\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:46:16.770603 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:16.770596    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fc6cfad-c77d-49cf-b6f1-fb4e4466a335-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:46:17.576660 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:17.576574    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"
Sep 11 13:46:17.598436 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:17.598406    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"]
Sep 11 13:46:17.601154 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:17.601131    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c0f589fb-558484f778-59zcl"]
Sep 11 13:46:19.316599 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:19.316565    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fc6cfad-c77d-49cf-b6f1-fb4e4466a335" path="/var/lib/kubelet/pods/7fc6cfad-c77d-49cf-b6f1-fb4e4466a335/volumes"
Sep 11 13:46:21.393100 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:21.393068    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:46:21.399138 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:21.399115    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:46:23.684572 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:23.684540    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"]
Sep 11 13:46:24.993766 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:24.993733    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"]
Sep 11 13:46:38.703529 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:38.703489    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc" podUID="90689975-f2c9-4806-8ec5-32eb740022f8" containerName="main" containerID="cri-o://0c0fc6bd96dd8e9e3ef7d8346029c7b4de48ece33a916cfc8aeab8697d0c4f23" gracePeriod=45
Sep 11 13:46:38.942427 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:38.942401    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:46:39.093789 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.093756    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-tmp-dir\") pod \"90689975-f2c9-4806-8ec5-32eb740022f8\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") "
Sep 11 13:46:39.093789 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.093794    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-home\") pod \"90689975-f2c9-4806-8ec5-32eb740022f8\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") "
Sep 11 13:46:39.094068 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.093820    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-model-cache\") pod \"90689975-f2c9-4806-8ec5-32eb740022f8\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") "
Sep 11 13:46:39.094068 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.093861    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90689975-f2c9-4806-8ec5-32eb740022f8-tls-certs\") pod \"90689975-f2c9-4806-8ec5-32eb740022f8\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") "
Sep 11 13:46:39.094068 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.093887    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgfs5\" (UniqueName: \"kubernetes.io/projected/90689975-f2c9-4806-8ec5-32eb740022f8-kube-api-access-jgfs5\") pod \"90689975-f2c9-4806-8ec5-32eb740022f8\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") "
Sep 11 13:46:39.094068 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.093964    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-dshm\") pod \"90689975-f2c9-4806-8ec5-32eb740022f8\" (UID: \"90689975-f2c9-4806-8ec5-32eb740022f8\") "
Sep 11 13:46:39.094285 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.094141    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "90689975-f2c9-4806-8ec5-32eb740022f8" (UID: "90689975-f2c9-4806-8ec5-32eb740022f8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:46:39.094285 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.094127    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-home" (OuterVolumeSpecName: "home") pod "90689975-f2c9-4806-8ec5-32eb740022f8" (UID: "90689975-f2c9-4806-8ec5-32eb740022f8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:46:39.094285 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.094149    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-model-cache" (OuterVolumeSpecName: "model-cache") pod "90689975-f2c9-4806-8ec5-32eb740022f8" (UID: "90689975-f2c9-4806-8ec5-32eb740022f8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:46:39.094395 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.094359    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:46:39.094395 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.094377    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:46:39.094395 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.094391    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:46:39.096232 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.096202    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90689975-f2c9-4806-8ec5-32eb740022f8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "90689975-f2c9-4806-8ec5-32eb740022f8" (UID: "90689975-f2c9-4806-8ec5-32eb740022f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:46:39.096344 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.096244    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-dshm" (OuterVolumeSpecName: "dshm") pod "90689975-f2c9-4806-8ec5-32eb740022f8" (UID: "90689975-f2c9-4806-8ec5-32eb740022f8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:46:39.096344 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.096251    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90689975-f2c9-4806-8ec5-32eb740022f8-kube-api-access-jgfs5" (OuterVolumeSpecName: "kube-api-access-jgfs5") pod "90689975-f2c9-4806-8ec5-32eb740022f8" (UID: "90689975-f2c9-4806-8ec5-32eb740022f8"). InnerVolumeSpecName "kube-api-access-jgfs5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:46:39.195784 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.195747    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90689975-f2c9-4806-8ec5-32eb740022f8-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:46:39.195784 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.195779    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgfs5\" (UniqueName: \"kubernetes.io/projected/90689975-f2c9-4806-8ec5-32eb740022f8-kube-api-access-jgfs5\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:46:39.195784 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.195789    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90689975-f2c9-4806-8ec5-32eb740022f8-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:46:39.713441 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.713408    2580 generic.go:356] "Generic (PLEG): container finished" podID="90689975-f2c9-4806-8ec5-32eb740022f8" containerID="0c0fc6bd96dd8e9e3ef7d8346029c7b4de48ece33a916cfc8aeab8697d0c4f23" exitCode=0
Sep 11 13:46:39.713949 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.713462    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"
Sep 11 13:46:39.713949 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.713499    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc" event={"ID":"90689975-f2c9-4806-8ec5-32eb740022f8","Type":"ContainerDied","Data":"0c0fc6bd96dd8e9e3ef7d8346029c7b4de48ece33a916cfc8aeab8697d0c4f23"}
Sep 11 13:46:39.713949 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.713537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc" event={"ID":"90689975-f2c9-4806-8ec5-32eb740022f8","Type":"ContainerDied","Data":"cf448fdab7b4ab4100c5a7352178006550292eb75c2ca9a0d539107d1e7fc285"}
Sep 11 13:46:39.713949 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.713555    2580 scope.go:117] "RemoveContainer" containerID="0c0fc6bd96dd8e9e3ef7d8346029c7b4de48ece33a916cfc8aeab8697d0c4f23"
Sep 11 13:46:39.722985 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.722967    2580 scope.go:117] "RemoveContainer" containerID="0c0fc6bd96dd8e9e3ef7d8346029c7b4de48ece33a916cfc8aeab8697d0c4f23"
Sep 11 13:46:39.723259 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:46:39.723241    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c0fc6bd96dd8e9e3ef7d8346029c7b4de48ece33a916cfc8aeab8697d0c4f23\": container with ID starting with 0c0fc6bd96dd8e9e3ef7d8346029c7b4de48ece33a916cfc8aeab8697d0c4f23 not found: ID does not exist" containerID="0c0fc6bd96dd8e9e3ef7d8346029c7b4de48ece33a916cfc8aeab8697d0c4f23"
Sep 11 13:46:39.723308 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.723266    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c0fc6bd96dd8e9e3ef7d8346029c7b4de48ece33a916cfc8aeab8697d0c4f23"} err="failed to get container status \"0c0fc6bd96dd8e9e3ef7d8346029c7b4de48ece33a916cfc8aeab8697d0c4f23\": rpc error: code = NotFound desc = could not find container \"0c0fc6bd96dd8e9e3ef7d8346029c7b4de48ece33a916cfc8aeab8697d0c4f23\": container with ID starting with 0c0fc6bd96dd8e9e3ef7d8346029c7b4de48ece33a916cfc8aeab8697d0c4f23 not found: ID does not exist"
Sep 11 13:46:39.745870 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.745844    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"]
Sep 11 13:46:39.756324 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:39.756305    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/e2e-conflict-b-f8ce9a34-kserve-5bd6bd4dbb-rjbjc"]
Sep 11 13:46:40.013378 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.013286    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z" podUID="a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d" containerName="main" containerID="cri-o://50206806b8ffa3724a7fd8096d29c18f8032209922f2e1d5a9a21852d9daaee2" gracePeriod=45
Sep 11 13:46:40.267006 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.266946    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:46:40.410003 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.409971    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-tls-certs\") pod \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") "
Sep 11 13:46:40.410171 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.410078    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-dshm\") pod \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") "
Sep 11 13:46:40.410171 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.410119    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l227\" (UniqueName: \"kubernetes.io/projected/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-kube-api-access-2l227\") pod \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") "
Sep 11 13:46:40.410171 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.410147    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-home\") pod \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") "
Sep 11 13:46:40.410288 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.410188    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-tmp-dir\") pod \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") "
Sep 11 13:46:40.410288 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.410221    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-model-cache\") pod \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\" (UID: \"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d\") "
Sep 11 13:46:40.410524 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.410491    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d" (UID: "a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:46:40.410524 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.410485    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-home" (OuterVolumeSpecName: "home") pod "a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d" (UID: "a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:46:40.410727 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.410558    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-model-cache" (OuterVolumeSpecName: "model-cache") pod "a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d" (UID: "a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:46:40.411036 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.411011    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:46:40.411140 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.411039    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:46:40.411140 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.411055    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:46:40.412466 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.412442    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-kube-api-access-2l227" (OuterVolumeSpecName: "kube-api-access-2l227") pod "a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d" (UID: "a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d"). InnerVolumeSpecName "kube-api-access-2l227". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:46:40.412466 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.412452    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d" (UID: "a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:46:40.412585 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.412535    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-dshm" (OuterVolumeSpecName: "dshm") pod "a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d" (UID: "a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:46:40.511618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.511589    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:46:40.511618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.511617    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l227\" (UniqueName: \"kubernetes.io/projected/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-kube-api-access-2l227\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:46:40.511618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.511627    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:46:40.722425 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.722393    2580 generic.go:356] "Generic (PLEG): container finished" podID="a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d" containerID="50206806b8ffa3724a7fd8096d29c18f8032209922f2e1d5a9a21852d9daaee2" exitCode=0
Sep 11 13:46:40.722836 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.722437    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"
Sep 11 13:46:40.722836 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.722476    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z" event={"ID":"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d","Type":"ContainerDied","Data":"50206806b8ffa3724a7fd8096d29c18f8032209922f2e1d5a9a21852d9daaee2"}
Sep 11 13:46:40.722836 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.722514    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z" event={"ID":"a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d","Type":"ContainerDied","Data":"7cc5588fde2fa9ea40755ac3a024df132f9fb42588cb2770c45e967cf5186102"}
Sep 11 13:46:40.722836 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.722531    2580 scope.go:117] "RemoveContainer" containerID="50206806b8ffa3724a7fd8096d29c18f8032209922f2e1d5a9a21852d9daaee2"
Sep 11 13:46:40.733298 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.733280    2580 scope.go:117] "RemoveContainer" containerID="50206806b8ffa3724a7fd8096d29c18f8032209922f2e1d5a9a21852d9daaee2"
Sep 11 13:46:40.733560 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:46:40.733543    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50206806b8ffa3724a7fd8096d29c18f8032209922f2e1d5a9a21852d9daaee2\": container with ID starting with 50206806b8ffa3724a7fd8096d29c18f8032209922f2e1d5a9a21852d9daaee2 not found: ID does not exist" containerID="50206806b8ffa3724a7fd8096d29c18f8032209922f2e1d5a9a21852d9daaee2"
Sep 11 13:46:40.733607 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.733568    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50206806b8ffa3724a7fd8096d29c18f8032209922f2e1d5a9a21852d9daaee2"} err="failed to get container status \"50206806b8ffa3724a7fd8096d29c18f8032209922f2e1d5a9a21852d9daaee2\": rpc error: code = NotFound desc = could not find container \"50206806b8ffa3724a7fd8096d29c18f8032209922f2e1d5a9a21852d9daaee2\": container with ID starting with 50206806b8ffa3724a7fd8096d29c18f8032209922f2e1d5a9a21852d9daaee2 not found: ID does not exist"
Sep 11 13:46:40.751143 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.751111    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"]
Sep 11 13:46:40.763793 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:40.763764    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-c0f589fb/e2e-lifecycle-model-c0f589fb-kserve-694558b5f5-wns6z"]
Sep 11 13:46:41.316645 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:41.316610    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90689975-f2c9-4806-8ec5-32eb740022f8" path="/var/lib/kubelet/pods/90689975-f2c9-4806-8ec5-32eb740022f8/volumes"
Sep 11 13:46:41.317164 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:41.317149    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d" path="/var/lib/kubelet/pods/a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d/volumes"
Sep 11 13:46:52.410559 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:46:52.410511    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:47:03.774027 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.773987    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"]
Sep 11 13:47:03.776270 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.774400    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d" containerName="main"
Sep 11 13:47:03.776270 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.774411    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d" containerName="main"
Sep 11 13:47:03.776270 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.774418    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90689975-f2c9-4806-8ec5-32eb740022f8" containerName="main"
Sep 11 13:47:03.776270 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.774424    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="90689975-f2c9-4806-8ec5-32eb740022f8" containerName="main"
Sep 11 13:47:03.776270 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.774442    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fc6cfad-c77d-49cf-b6f1-fb4e4466a335" containerName="payload-pre-processing"
Sep 11 13:47:03.776270 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.774448    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fc6cfad-c77d-49cf-b6f1-fb4e4466a335" containerName="payload-pre-processing"
Sep 11 13:47:03.776270 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.774514    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a96ccaa4-844a-4b38-9bd8-0c11bbcebb5d" containerName="main"
Sep 11 13:47:03.776270 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.774522    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="90689975-f2c9-4806-8ec5-32eb740022f8" containerName="main"
Sep 11 13:47:03.776270 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.774529    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fc6cfad-c77d-49cf-b6f1-fb4e4466a335" containerName="payload-pre-processing"
Sep 11 13:47:03.777543 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.777526    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.780376 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.780351    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-371c38\"/\"distinct-w4-371c38-kserve-self-signed-certs\""
Sep 11 13:47:03.800981 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.800951    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"]
Sep 11 13:47:03.820110 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.820086    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-tmp-dir\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.820234 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.820118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-home\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.820234 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.820201    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz7mr\" (UniqueName: \"kubernetes.io/projected/734aa3c9-314b-4d19-ae75-6c6469ae244a-kube-api-access-lz7mr\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.820234 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.820228    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-dshm\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.820344 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.820266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/734aa3c9-314b-4d19-ae75-6c6469ae244a-tls-certs\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.820344 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.820312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-model-cache\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.921483 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.921433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-model-cache\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.921657 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.921629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-tmp-dir\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.921703 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.921659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-home\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.921751 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.921737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz7mr\" (UniqueName: \"kubernetes.io/projected/734aa3c9-314b-4d19-ae75-6c6469ae244a-kube-api-access-lz7mr\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.921787 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.921760    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-dshm\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.921922 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.921881    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-model-cache\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.922085 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.921933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/734aa3c9-314b-4d19-ae75-6c6469ae244a-tls-certs\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.922168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.921968    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-tmp-dir\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.922222 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.922188    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-home\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.924076 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.924055    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-dshm\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.924467 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.924447    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/734aa3c9-314b-4d19-ae75-6c6469ae244a-tls-certs\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:03.947470 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:03.947442    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz7mr\" (UniqueName: \"kubernetes.io/projected/734aa3c9-314b-4d19-ae75-6c6469ae244a-kube-api-access-lz7mr\") pod \"distinct-w4-371c38-kserve-587f4ddf8b-cx726\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:04.087004 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:04.086972    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:04.389846 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:04.389752    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"]
Sep 11 13:47:04.393866 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:47:04.393839    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod734aa3c9_314b_4d19_ae75_6c6469ae244a.slice/crio-7460184e3e52d4a50720c65144341e77a5061d21ae756c459bf2f600f4af2f4e WatchSource:0}: Error finding container 7460184e3e52d4a50720c65144341e77a5061d21ae756c459bf2f600f4af2f4e: Status 404 returned error can't find the container with id 7460184e3e52d4a50720c65144341e77a5061d21ae756c459bf2f600f4af2f4e
Sep 11 13:47:04.869291 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:04.869252    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726" event={"ID":"734aa3c9-314b-4d19-ae75-6c6469ae244a","Type":"ContainerStarted","Data":"cfd102db5934572cb8c9246d73609fa360b713fa6a991fbf3512f5076efe6114"}
Sep 11 13:47:04.869669 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:04.869301    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726" event={"ID":"734aa3c9-314b-4d19-ae75-6c6469ae244a","Type":"ContainerStarted","Data":"7460184e3e52d4a50720c65144341e77a5061d21ae756c459bf2f600f4af2f4e"}
Sep 11 13:47:04.914880 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:04.914824    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726" podStartSLOduration=1.9148058159999999 podStartE2EDuration="1.914805816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:47:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:47:04.91379756 +0000 UTC m=+1544.132335055" watchObservedRunningTime="2026-09-11 13:47:04.914805816 +0000 UTC m=+1544.133343307"
Sep 11 13:47:14.087781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:14.087742    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:14.088235 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:14.087802    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:14.100461 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:14.100433    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:14.940993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:14.940953    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:47:29.624033 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:47:29.624000    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:48:05.410088 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:05.410049    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:48:37.205283 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:37.205196    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:48:51.359596 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.359562    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"]
Sep 11 13:48:51.363650 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.363631    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.366326 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.366304    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-082f62\"/\"embedding-w0-082f62-kserve-self-signed-certs\""
Sep 11 13:48:51.370020 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.369992    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-home\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.370278 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.370256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c57f759-a7ac-457d-8233-6bc2be4dc31d-tls-certs\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.371051 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.370496    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-model-cache\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.371051 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.370544    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t67px\" (UniqueName: \"kubernetes.io/projected/4c57f759-a7ac-457d-8233-6bc2be4dc31d-kube-api-access-t67px\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.371051 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.370652    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-dshm\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.371051 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.370751    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-tmp-dir\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.374576 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.374552    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"]
Sep 11 13:48:51.471186 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.471152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-tmp-dir\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.471372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.471205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-home\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.471372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.471240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c57f759-a7ac-457d-8233-6bc2be4dc31d-tls-certs\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.471372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.471278    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-model-cache\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.471372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.471294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t67px\" (UniqueName: \"kubernetes.io/projected/4c57f759-a7ac-457d-8233-6bc2be4dc31d-kube-api-access-t67px\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.471372 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.471319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-dshm\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.471679 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.471627    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-tmp-dir\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.471679 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.471638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-home\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.471788 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.471702    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-model-cache\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.473572 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.473547    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-dshm\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.473825 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.473807    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c57f759-a7ac-457d-8233-6bc2be4dc31d-tls-certs\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.480186 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.480168    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t67px\" (UniqueName: \"kubernetes.io/projected/4c57f759-a7ac-457d-8233-6bc2be4dc31d-kube-api-access-t67px\") pod \"embedding-w0-082f62-kserve-59457d844-l9nbr\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") " pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.673512 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.673413    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:48:51.950082 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:51.949744    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"]
Sep 11 13:48:51.953292 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:48:51.953259    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c57f759_a7ac_457d_8233_6bc2be4dc31d.slice/crio-fe652245a350de6e370dc1b8dea0d25bc7a958e380705a5c6c00a439334e74ae WatchSource:0}: Error finding container fe652245a350de6e370dc1b8dea0d25bc7a958e380705a5c6c00a439334e74ae: Status 404 returned error can't find the container with id fe652245a350de6e370dc1b8dea0d25bc7a958e380705a5c6c00a439334e74ae
Sep 11 13:48:52.542853 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:52.542804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr" event={"ID":"4c57f759-a7ac-457d-8233-6bc2be4dc31d","Type":"ContainerStarted","Data":"b4e6bebf805e58a2f64ebe31c79e8d22ce1864d5878aad949a883b475c98b8df"}
Sep 11 13:48:52.542853 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:52.542859    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr" event={"ID":"4c57f759-a7ac-457d-8233-6bc2be4dc31d","Type":"ContainerStarted","Data":"fe652245a350de6e370dc1b8dea0d25bc7a958e380705a5c6c00a439334e74ae"}
Sep 11 13:48:52.570964 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:48:52.570883    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr" podStartSLOduration=1.570865499 podStartE2EDuration="1.570865499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:48:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:52.568074502 +0000 UTC m=+1651.786611994" watchObservedRunningTime="2026-09-11 13:48:52.570865499 +0000 UTC m=+1651.789403020"
Sep 11 13:49:01.674009 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:01.673968    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:49:01.674009 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:01.674020    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:49:01.686819 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:01.686786    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:49:02.615310 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:02.615276    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:49:11.580556 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:11.580519    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"]
Sep 11 13:49:11.584454 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:11.584436    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"
Sep 11 13:49:11.589459 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:11.589437    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-c8d2f6-dockercfg-j8n8g\""
Sep 11 13:49:11.590600 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:11.590580    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-c8d2f6\""
Sep 11 13:49:11.605132 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:11.605108    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"]
Sep 11 13:49:11.695782 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:11.695751    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp4gf\" (UniqueName: \"kubernetes.io/projected/123f0cac-7e98-4fce-a9fe-28584e2f4dfe-kube-api-access-rp4gf\") pod \"payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7\" (UID: \"123f0cac-7e98-4fce-a9fe-28584e2f4dfe\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"
Sep 11 13:49:11.695782 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:11.695784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/123f0cac-7e98-4fce-a9fe-28584e2f4dfe-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7\" (UID: \"123f0cac-7e98-4fce-a9fe-28584e2f4dfe\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"
Sep 11 13:49:11.796846 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:11.796815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp4gf\" (UniqueName: \"kubernetes.io/projected/123f0cac-7e98-4fce-a9fe-28584e2f4dfe-kube-api-access-rp4gf\") pod \"payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7\" (UID: \"123f0cac-7e98-4fce-a9fe-28584e2f4dfe\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"
Sep 11 13:49:11.796846 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:11.796852    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/123f0cac-7e98-4fce-a9fe-28584e2f4dfe-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7\" (UID: \"123f0cac-7e98-4fce-a9fe-28584e2f4dfe\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"
Sep 11 13:49:11.797637 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:11.797617    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/123f0cac-7e98-4fce-a9fe-28584e2f4dfe-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7\" (UID: \"123f0cac-7e98-4fce-a9fe-28584e2f4dfe\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"
Sep 11 13:49:11.809327 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:11.809299    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp4gf\" (UniqueName: \"kubernetes.io/projected/123f0cac-7e98-4fce-a9fe-28584e2f4dfe-kube-api-access-rp4gf\") pod \"payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7\" (UID: \"123f0cac-7e98-4fce-a9fe-28584e2f4dfe\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"
Sep 11 13:49:11.895577 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:11.895483    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"
Sep 11 13:49:12.177492 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:12.177460    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"]
Sep 11 13:49:12.178122 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:49:12.178098    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod123f0cac_7e98_4fce_a9fe_28584e2f4dfe.slice/crio-e3b2c79eb777def4434a5be69714bf7b5104d5c36f64cf0fdaf776e4480c9f7b WatchSource:0}: Error finding container e3b2c79eb777def4434a5be69714bf7b5104d5c36f64cf0fdaf776e4480c9f7b: Status 404 returned error can't find the container with id e3b2c79eb777def4434a5be69714bf7b5104d5c36f64cf0fdaf776e4480c9f7b
Sep 11 13:49:12.666284 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:12.666248    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7" event={"ID":"123f0cac-7e98-4fce-a9fe-28584e2f4dfe","Type":"ContainerStarted","Data":"eaa59d713c1e7c82c5db0729c04ceb719c86bdfd5daceaebea513e95d5e88738"}
Sep 11 13:49:12.666671 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:12.666304    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7" event={"ID":"123f0cac-7e98-4fce-a9fe-28584e2f4dfe","Type":"ContainerStarted","Data":"e3b2c79eb777def4434a5be69714bf7b5104d5c36f64cf0fdaf776e4480c9f7b"}
Sep 11 13:49:12.666671 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:12.666440    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"
Sep 11 13:49:12.694419 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:12.694371    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7" podStartSLOduration=1.6943544130000001 podStartE2EDuration="1.694354413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:12.693097843 +0000 UTC m=+1671.911635337" watchObservedRunningTime="2026-09-11 13:49:12.694354413 +0000 UTC m=+1671.912891941"
Sep 11 13:49:21.590741 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:21.590709    2580 scope.go:117] "RemoveContainer" containerID="f400c0f1e521f41fe8f6b2e90bb321cc4d76185dcba60e8c79abca0fc810feca"
Sep 11 13:49:23.674831 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:23.674803    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"
Sep 11 13:49:25.041204 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:25.041169    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0a2395-5ll94"]
Sep 11 13:49:25.044746 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:25.044729    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0a2395-5ll94"
Sep 11 13:49:25.047606 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:25.047584    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:49:25.047737 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:25.047692    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:49:25.053374 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:25.053352    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0a2395-5ll94"]
Sep 11 13:49:25.131411 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:25.131369    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-0a2395-5ll94\" (UID: \"cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0a2395-5ll94"
Sep 11 13:49:25.232119 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:25.232088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-0a2395-5ll94\" (UID: \"cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0a2395-5ll94"
Sep 11 13:49:25.232751 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:25.232723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-0a2395-5ll94\" (UID: \"cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0a2395-5ll94"
Sep 11 13:49:25.357436 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:25.357394    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0a2395-5ll94"
Sep 11 13:49:25.641430 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:25.641402    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0a2395-5ll94"]
Sep 11 13:49:25.642072 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:49:25.642046    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb04c202_1bf0_4a4b_9a9c_8bcdfe93d984.slice/crio-1629cc9428f725d3db5682dbd2df718856476d9d8226e0b1e8d4f89d4f6f253e WatchSource:0}: Error finding container 1629cc9428f725d3db5682dbd2df718856476d9d8226e0b1e8d4f89d4f6f253e: Status 404 returned error can't find the container with id 1629cc9428f725d3db5682dbd2df718856476d9d8226e0b1e8d4f89d4f6f253e
Sep 11 13:49:25.748168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:25.748127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0a2395-5ll94" event={"ID":"cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984","Type":"ContainerStarted","Data":"1629cc9428f725d3db5682dbd2df718856476d9d8226e0b1e8d4f89d4f6f253e"}
Sep 11 13:49:26.755859 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:26.755764    2580 generic.go:356] "Generic (PLEG): container finished" podID="cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984" containerID="3cdc1510e65ddaea0ecf318915ab7e39e3e52581bf8218660766e737b4842a91" exitCode=0
Sep 11 13:49:26.756263 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:26.755912    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0a2395-5ll94" event={"ID":"cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984","Type":"ContainerDied","Data":"3cdc1510e65ddaea0ecf318915ab7e39e3e52581bf8218660766e737b4842a91"}
Sep 11 13:49:27.908090 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:27.908064    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0a2395-5ll94"
Sep 11 13:49:27.960926 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:27.960851    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984-maas-api-service-ca\") pod \"cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984\" (UID: \"cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984\") "
Sep 11 13:49:27.961215 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:27.961188    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984" (UID: "cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:49:28.062100 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:28.062073    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984-maas-api-service-ca\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:49:28.769368 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:28.769335    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0a2395-5ll94"
Sep 11 13:49:28.769549 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:28.769390    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0a2395-5ll94" event={"ID":"cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984","Type":"ContainerDied","Data":"1629cc9428f725d3db5682dbd2df718856476d9d8226e0b1e8d4f89d4f6f253e"}
Sep 11 13:49:28.769549 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:28.769437    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1629cc9428f725d3db5682dbd2df718856476d9d8226e0b1e8d4f89d4f6f253e"
Sep 11 13:49:42.283355 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:42.283316    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0a2395-5ll94"]
Sep 11 13:49:42.297929 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:42.297869    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-0a2395-5ll94"]
Sep 11 13:49:43.317548 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:43.317474    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984" path="/var/lib/kubelet/pods/cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984/volumes"
Sep 11 13:49:51.097701 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:51.097657    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5"]
Sep 11 13:49:51.098136 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:51.098122    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984" containerName="revoke-keys"
Sep 11 13:49:51.098190 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:51.098138    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984" containerName="revoke-keys"
Sep 11 13:49:51.098229 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:51.098217    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb04c202-1bf0-4a4b-9a9c-8bcdfe93d984" containerName="revoke-keys"
Sep 11 13:49:51.101598 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:51.101580    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5"
Sep 11 13:49:51.104705 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:51.104677    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:49:51.105013 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:51.104989    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:49:51.111930 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:51.111894    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5"]
Sep 11 13:49:51.202321 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:51.202282    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091a1b64-dc99-444b-81eb-f0f7ec8b09e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5\" (UID: \"091a1b64-dc99-444b-81eb-f0f7ec8b09e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5"
Sep 11 13:49:51.303116 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:51.303069    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091a1b64-dc99-444b-81eb-f0f7ec8b09e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5\" (UID: \"091a1b64-dc99-444b-81eb-f0f7ec8b09e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5"
Sep 11 13:49:51.303686 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:51.303664    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091a1b64-dc99-444b-81eb-f0f7ec8b09e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5\" (UID: \"091a1b64-dc99-444b-81eb-f0f7ec8b09e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5"
Sep 11 13:49:51.414254 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:51.414162    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5"
Sep 11 13:49:51.692092 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:51.692017    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5"]
Sep 11 13:49:51.695503 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:49:51.695461    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091a1b64_dc99_444b_81eb_f0f7ec8b09e8.slice/crio-45ef1daea3b00de4cf0950b2e194a3b2c680408e9a903398eee0d369b9ccd9c2 WatchSource:0}: Error finding container 45ef1daea3b00de4cf0950b2e194a3b2c680408e9a903398eee0d369b9ccd9c2: Status 404 returned error can't find the container with id 45ef1daea3b00de4cf0950b2e194a3b2c680408e9a903398eee0d369b9ccd9c2
Sep 11 13:49:51.913054 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:51.913019    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5" event={"ID":"091a1b64-dc99-444b-81eb-f0f7ec8b09e8","Type":"ContainerStarted","Data":"45ef1daea3b00de4cf0950b2e194a3b2c680408e9a903398eee0d369b9ccd9c2"}
Sep 11 13:49:52.920870 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:52.920833    2580 generic.go:356] "Generic (PLEG): container finished" podID="091a1b64-dc99-444b-81eb-f0f7ec8b09e8" containerID="658b66883cd336f596831017e96de0b920767d3ec49fec643bdcbfb63295a68a" exitCode=0
Sep 11 13:49:52.921279 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:52.920878    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5" event={"ID":"091a1b64-dc99-444b-81eb-f0f7ec8b09e8","Type":"ContainerDied","Data":"658b66883cd336f596831017e96de0b920767d3ec49fec643bdcbfb63295a68a"}
Sep 11 13:49:54.070672 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:54.070644    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5"
Sep 11 13:49:54.130493 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:54.130461    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091a1b64-dc99-444b-81eb-f0f7ec8b09e8-maas-api-service-ca\") pod \"091a1b64-dc99-444b-81eb-f0f7ec8b09e8\" (UID: \"091a1b64-dc99-444b-81eb-f0f7ec8b09e8\") "
Sep 11 13:49:54.130894 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:54.130871    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/091a1b64-dc99-444b-81eb-f0f7ec8b09e8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "091a1b64-dc99-444b-81eb-f0f7ec8b09e8" (UID: "091a1b64-dc99-444b-81eb-f0f7ec8b09e8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:49:54.231432 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:54.231352    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/091a1b64-dc99-444b-81eb-f0f7ec8b09e8-maas-api-service-ca\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:49:54.936679 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:54.936644    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5" event={"ID":"091a1b64-dc99-444b-81eb-f0f7ec8b09e8","Type":"ContainerDied","Data":"45ef1daea3b00de4cf0950b2e194a3b2c680408e9a903398eee0d369b9ccd9c2"}
Sep 11 13:49:54.936679 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:54.936669    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5"
Sep 11 13:49:54.936881 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:49:54.936697    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45ef1daea3b00de4cf0950b2e194a3b2c680408e9a903398eee0d369b9ccd9c2"
Sep 11 13:50:05.209068 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:05.209028    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:50:06.945841 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:06.945807    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"]
Sep 11 13:50:06.946309 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:06.946047    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7" podUID="123f0cac-7e98-4fce-a9fe-28584e2f4dfe" containerName="payload-pre-processing" containerID="cri-o://eaa59d713c1e7c82c5db0729c04ceb719c86bdfd5daceaebea513e95d5e88738" gracePeriod=30
Sep 11 13:50:06.978841 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:50:06.978806    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod123f0cac_7e98_4fce_a9fe_28584e2f4dfe.slice/crio-eaa59d713c1e7c82c5db0729c04ceb719c86bdfd5daceaebea513e95d5e88738.scope\": RecentStats: unable to find data in memory cache]"
Sep 11 13:50:07.201479 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:07.201416    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"
Sep 11 13:50:07.251275 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:07.251238    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5"]
Sep 11 13:50:07.257597 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:07.257571    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rp4gf\" (UniqueName: \"kubernetes.io/projected/123f0cac-7e98-4fce-a9fe-28584e2f4dfe-kube-api-access-rp4gf\") pod \"123f0cac-7e98-4fce-a9fe-28584e2f4dfe\" (UID: \"123f0cac-7e98-4fce-a9fe-28584e2f4dfe\") "
Sep 11 13:50:07.257756 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:07.257728    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/123f0cac-7e98-4fce-a9fe-28584e2f4dfe-plugins-config-volume\") pod \"123f0cac-7e98-4fce-a9fe-28584e2f4dfe\" (UID: \"123f0cac-7e98-4fce-a9fe-28584e2f4dfe\") "
Sep 11 13:50:07.258152 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:07.258121    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/123f0cac-7e98-4fce-a9fe-28584e2f4dfe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "123f0cac-7e98-4fce-a9fe-28584e2f4dfe" (UID: "123f0cac-7e98-4fce-a9fe-28584e2f4dfe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:07.258523 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:07.258502    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-c8d2f6-9zdj5"]
Sep 11 13:50:07.259888 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:07.259859    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/123f0cac-7e98-4fce-a9fe-28584e2f4dfe-kube-api-access-rp4gf" (OuterVolumeSpecName: "kube-api-access-rp4gf") pod "123f0cac-7e98-4fce-a9fe-28584e2f4dfe" (UID: "123f0cac-7e98-4fce-a9fe-28584e2f4dfe"). InnerVolumeSpecName "kube-api-access-rp4gf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:07.316041 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:07.316010    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="091a1b64-dc99-444b-81eb-f0f7ec8b09e8" path="/var/lib/kubelet/pods/091a1b64-dc99-444b-81eb-f0f7ec8b09e8/volumes"
Sep 11 13:50:07.358796 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:07.358756    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/123f0cac-7e98-4fce-a9fe-28584e2f4dfe-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:07.358796 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:07.358787    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp4gf\" (UniqueName: \"kubernetes.io/projected/123f0cac-7e98-4fce-a9fe-28584e2f4dfe-kube-api-access-rp4gf\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:08.018609 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:08.018571    2580 generic.go:356] "Generic (PLEG): container finished" podID="123f0cac-7e98-4fce-a9fe-28584e2f4dfe" containerID="eaa59d713c1e7c82c5db0729c04ceb719c86bdfd5daceaebea513e95d5e88738" exitCode=0
Sep 11 13:50:08.019041 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:08.018617    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"
Sep 11 13:50:08.019041 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:08.018654    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7" event={"ID":"123f0cac-7e98-4fce-a9fe-28584e2f4dfe","Type":"ContainerDied","Data":"eaa59d713c1e7c82c5db0729c04ceb719c86bdfd5daceaebea513e95d5e88738"}
Sep 11 13:50:08.019041 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:08.018712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7" event={"ID":"123f0cac-7e98-4fce-a9fe-28584e2f4dfe","Type":"ContainerDied","Data":"e3b2c79eb777def4434a5be69714bf7b5104d5c36f64cf0fdaf776e4480c9f7b"}
Sep 11 13:50:08.019041 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:08.018735    2580 scope.go:117] "RemoveContainer" containerID="eaa59d713c1e7c82c5db0729c04ceb719c86bdfd5daceaebea513e95d5e88738"
Sep 11 13:50:08.029174 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:08.029157    2580 scope.go:117] "RemoveContainer" containerID="eaa59d713c1e7c82c5db0729c04ceb719c86bdfd5daceaebea513e95d5e88738"
Sep 11 13:50:08.030489 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:50:08.029693    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaa59d713c1e7c82c5db0729c04ceb719c86bdfd5daceaebea513e95d5e88738\": container with ID starting with eaa59d713c1e7c82c5db0729c04ceb719c86bdfd5daceaebea513e95d5e88738 not found: ID does not exist" containerID="eaa59d713c1e7c82c5db0729c04ceb719c86bdfd5daceaebea513e95d5e88738"
Sep 11 13:50:08.030489 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:08.029725    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaa59d713c1e7c82c5db0729c04ceb719c86bdfd5daceaebea513e95d5e88738"} err="failed to get container status \"eaa59d713c1e7c82c5db0729c04ceb719c86bdfd5daceaebea513e95d5e88738\": rpc error: code = NotFound desc = could not find container \"eaa59d713c1e7c82c5db0729c04ceb719c86bdfd5daceaebea513e95d5e88738\": container with ID starting with eaa59d713c1e7c82c5db0729c04ceb719c86bdfd5daceaebea513e95d5e88738 not found: ID does not exist"
Sep 11 13:50:08.054301 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:08.054260    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"]
Sep 11 13:50:08.057082 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:08.057060    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c8d2f6-79fb4b9f4-cvjd7"]
Sep 11 13:50:09.316214 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:09.316181    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="123f0cac-7e98-4fce-a9fe-28584e2f4dfe" path="/var/lib/kubelet/pods/123f0cac-7e98-4fce-a9fe-28584e2f4dfe/volumes"
Sep 11 13:50:21.614528 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:21.614495    2580 scope.go:117] "RemoveContainer" containerID="cccc576c552c307a081d5d527c87adea837edd6f41d45c4fc4ff14307844a65b"
Sep 11 13:50:27.106799 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:27.106765    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-470eeb-29818905-lpk2d"]
Sep 11 13:50:27.113666 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:27.113636    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-470eeb-29818905-lpk2d"]
Sep 11 13:50:27.316101 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:27.316060    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="893c3866-dd59-48d6-8b25-4d1c832bea7c" path="/var/lib/kubelet/pods/893c3866-dd59-48d6-8b25-4d1c832bea7c/volumes"
Sep 11 13:50:37.951988 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:37.951952    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb"]
Sep 11 13:50:37.952426 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:37.952411    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091a1b64-dc99-444b-81eb-f0f7ec8b09e8" containerName="revoke-keys"
Sep 11 13:50:37.952473 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:37.952428    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="091a1b64-dc99-444b-81eb-f0f7ec8b09e8" containerName="revoke-keys"
Sep 11 13:50:37.952473 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:37.952448    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="123f0cac-7e98-4fce-a9fe-28584e2f4dfe" containerName="payload-pre-processing"
Sep 11 13:50:37.952473 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:37.952454    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="123f0cac-7e98-4fce-a9fe-28584e2f4dfe" containerName="payload-pre-processing"
Sep 11 13:50:37.952578 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:37.952564    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="091a1b64-dc99-444b-81eb-f0f7ec8b09e8" containerName="revoke-keys"
Sep 11 13:50:37.952578 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:37.952575    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="123f0cac-7e98-4fce-a9fe-28584e2f4dfe" containerName="payload-pre-processing"
Sep 11 13:50:37.955956 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:37.955939    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb"
Sep 11 13:50:37.962576 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:37.962553    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:50:37.962576 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:37.962565    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:50:38.015960 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:38.015931    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb"]
Sep 11 13:50:38.075343 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:38.075310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db7221f8-224f-463e-851b-dbc29efa946d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb\" (UID: \"db7221f8-224f-463e-851b-dbc29efa946d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb"
Sep 11 13:50:38.177098 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:38.177068    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db7221f8-224f-463e-851b-dbc29efa946d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb\" (UID: \"db7221f8-224f-463e-851b-dbc29efa946d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb"
Sep 11 13:50:38.177752 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:38.177724    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db7221f8-224f-463e-851b-dbc29efa946d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb\" (UID: \"db7221f8-224f-463e-851b-dbc29efa946d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb"
Sep 11 13:50:38.266272 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:38.266176    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb"
Sep 11 13:50:38.553887 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:38.553861    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb"]
Sep 11 13:50:38.554718 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:50:38.554693    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb7221f8_224f_463e_851b_dbc29efa946d.slice/crio-d4f5d001f2db87a1e8f8dd40f5a3db75d971dc8f6fb540a795d374d8ed8c10f0 WatchSource:0}: Error finding container d4f5d001f2db87a1e8f8dd40f5a3db75d971dc8f6fb540a795d374d8ed8c10f0: Status 404 returned error can't find the container with id d4f5d001f2db87a1e8f8dd40f5a3db75d971dc8f6fb540a795d374d8ed8c10f0
Sep 11 13:50:39.214894 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:39.214800    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb" event={"ID":"db7221f8-224f-463e-851b-dbc29efa946d","Type":"ContainerStarted","Data":"a268c187e2f233245bfc256bcf67ae2ac171e7ed9e14d667299914cca541347d"}
Sep 11 13:50:39.215264 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:39.214897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb" event={"ID":"db7221f8-224f-463e-851b-dbc29efa946d","Type":"ContainerStarted","Data":"d4f5d001f2db87a1e8f8dd40f5a3db75d971dc8f6fb540a795d374d8ed8c10f0"}
Sep 11 13:50:39.247330 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:39.247281    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb" podStartSLOduration=2.247266568 podStartE2EDuration="2.247266568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:50:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:50:39.244363153 +0000 UTC m=+1758.462900644" watchObservedRunningTime="2026-09-11 13:50:39.247266568 +0000 UTC m=+1758.465804060"
Sep 11 13:50:40.223080 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:40.223043    2580 generic.go:356] "Generic (PLEG): container finished" podID="db7221f8-224f-463e-851b-dbc29efa946d" containerID="a268c187e2f233245bfc256bcf67ae2ac171e7ed9e14d667299914cca541347d" exitCode=0
Sep 11 13:50:40.223459 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:40.223164    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb" event={"ID":"db7221f8-224f-463e-851b-dbc29efa946d","Type":"ContainerDied","Data":"a268c187e2f233245bfc256bcf67ae2ac171e7ed9e14d667299914cca541347d"}
Sep 11 13:50:40.898204 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:40.898158    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"]
Sep 11 13:50:40.905228 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:40.905198    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"]
Sep 11 13:50:40.918491 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:40.918461    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"]
Sep 11 13:50:40.936198 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:40.936167    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"]
Sep 11 13:50:40.948675 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:40.948642    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"]
Sep 11 13:50:41.376312 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:41.376287    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb"
Sep 11 13:50:41.512022 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:41.511936    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db7221f8-224f-463e-851b-dbc29efa946d-maas-api-service-ca\") pod \"db7221f8-224f-463e-851b-dbc29efa946d\" (UID: \"db7221f8-224f-463e-851b-dbc29efa946d\") "
Sep 11 13:50:41.512361 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:41.512336    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db7221f8-224f-463e-851b-dbc29efa946d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "db7221f8-224f-463e-851b-dbc29efa946d" (UID: "db7221f8-224f-463e-851b-dbc29efa946d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:41.512420 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:41.512408    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db7221f8-224f-463e-851b-dbc29efa946d-maas-api-service-ca\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:42.236776 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:42.236751    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb"
Sep 11 13:50:42.236967 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:42.236753    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb" event={"ID":"db7221f8-224f-463e-851b-dbc29efa946d","Type":"ContainerDied","Data":"d4f5d001f2db87a1e8f8dd40f5a3db75d971dc8f6fb540a795d374d8ed8c10f0"}
Sep 11 13:50:42.236967 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:42.236868    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4f5d001f2db87a1e8f8dd40f5a3db75d971dc8f6fb540a795d374d8ed8c10f0"
Sep 11 13:50:48.782381 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:48.782341    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-e52aaa-29818905-8v4lm"]
Sep 11 13:50:48.785037 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:48.785004    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-e52aaa-29818905-8v4lm"]
Sep 11 13:50:48.826166 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:48.826132    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"]
Sep 11 13:50:48.826434 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:48.826401    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4" podUID="40658cad-e2d6-4d22-97f4-90a264ac2a8a" containerName="payload-pre-processing" containerID="cri-o://0f79bd90da5059f62c7441834566231677d103bf5c2005bd6f1ea1349e196c99" gracePeriod=30
Sep 11 13:50:49.037423 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.037344    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb"]
Sep 11 13:50:49.040200 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.040174    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e52aaa-vwjrb"]
Sep 11 13:50:49.089001 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.088977    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"
Sep 11 13:50:49.185939 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.185885    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40658cad-e2d6-4d22-97f4-90a264ac2a8a-plugins-config-volume\") pod \"40658cad-e2d6-4d22-97f4-90a264ac2a8a\" (UID: \"40658cad-e2d6-4d22-97f4-90a264ac2a8a\") "
Sep 11 13:50:49.186136 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.186005    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx56w\" (UniqueName: \"kubernetes.io/projected/40658cad-e2d6-4d22-97f4-90a264ac2a8a-kube-api-access-rx56w\") pod \"40658cad-e2d6-4d22-97f4-90a264ac2a8a\" (UID: \"40658cad-e2d6-4d22-97f4-90a264ac2a8a\") "
Sep 11 13:50:49.186299 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.186273    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40658cad-e2d6-4d22-97f4-90a264ac2a8a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "40658cad-e2d6-4d22-97f4-90a264ac2a8a" (UID: "40658cad-e2d6-4d22-97f4-90a264ac2a8a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:49.188081 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.188056    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40658cad-e2d6-4d22-97f4-90a264ac2a8a-kube-api-access-rx56w" (OuterVolumeSpecName: "kube-api-access-rx56w") pod "40658cad-e2d6-4d22-97f4-90a264ac2a8a" (UID: "40658cad-e2d6-4d22-97f4-90a264ac2a8a"). InnerVolumeSpecName "kube-api-access-rx56w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:49.282094 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.282061    2580 generic.go:356] "Generic (PLEG): container finished" podID="40658cad-e2d6-4d22-97f4-90a264ac2a8a" containerID="0f79bd90da5059f62c7441834566231677d103bf5c2005bd6f1ea1349e196c99" exitCode=0
Sep 11 13:50:49.282280 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.282105    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"
Sep 11 13:50:49.282280 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.282138    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4" event={"ID":"40658cad-e2d6-4d22-97f4-90a264ac2a8a","Type":"ContainerDied","Data":"0f79bd90da5059f62c7441834566231677d103bf5c2005bd6f1ea1349e196c99"}
Sep 11 13:50:49.282280 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.282188    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4" event={"ID":"40658cad-e2d6-4d22-97f4-90a264ac2a8a","Type":"ContainerDied","Data":"275f594d4fb70d5eb5e24be585f55e5274b0f0de203c91bad0093f60b1e35048"}
Sep 11 13:50:49.282280 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.282216    2580 scope.go:117] "RemoveContainer" containerID="0f79bd90da5059f62c7441834566231677d103bf5c2005bd6f1ea1349e196c99"
Sep 11 13:50:49.286992 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.286965    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx56w\" (UniqueName: \"kubernetes.io/projected/40658cad-e2d6-4d22-97f4-90a264ac2a8a-kube-api-access-rx56w\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:49.286992 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.286994    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40658cad-e2d6-4d22-97f4-90a264ac2a8a-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:49.294661 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.294638    2580 scope.go:117] "RemoveContainer" containerID="0f79bd90da5059f62c7441834566231677d103bf5c2005bd6f1ea1349e196c99"
Sep 11 13:50:49.295021 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:50:49.294991    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f79bd90da5059f62c7441834566231677d103bf5c2005bd6f1ea1349e196c99\": container with ID starting with 0f79bd90da5059f62c7441834566231677d103bf5c2005bd6f1ea1349e196c99 not found: ID does not exist" containerID="0f79bd90da5059f62c7441834566231677d103bf5c2005bd6f1ea1349e196c99"
Sep 11 13:50:49.295114 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.295029    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f79bd90da5059f62c7441834566231677d103bf5c2005bd6f1ea1349e196c99"} err="failed to get container status \"0f79bd90da5059f62c7441834566231677d103bf5c2005bd6f1ea1349e196c99\": rpc error: code = NotFound desc = could not find container \"0f79bd90da5059f62c7441834566231677d103bf5c2005bd6f1ea1349e196c99\": container with ID starting with 0f79bd90da5059f62c7441834566231677d103bf5c2005bd6f1ea1349e196c99 not found: ID does not exist"
Sep 11 13:50:49.316343 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.316310    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31489940-f821-4263-a128-fbe7aeeeccb4" path="/var/lib/kubelet/pods/31489940-f821-4263-a128-fbe7aeeeccb4/volumes"
Sep 11 13:50:49.316657 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.316635    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db7221f8-224f-463e-851b-dbc29efa946d" path="/var/lib/kubelet/pods/db7221f8-224f-463e-851b-dbc29efa946d/volumes"
Sep 11 13:50:49.316938 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.316924    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"]
Sep 11 13:50:49.321350 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:49.321318    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e52aaa-84cc978478-56ch4"]
Sep 11 13:50:51.315712 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:51.315678    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40658cad-e2d6-4d22-97f4-90a264ac2a8a" path="/var/lib/kubelet/pods/40658cad-e2d6-4d22-97f4-90a264ac2a8a/volumes"
Sep 11 13:50:52.184066 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:52.184032    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:50:53.330763 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.330733    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"]
Sep 11 13:50:53.331297 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.330933    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx" podUID="6bfc7841-f1da-4e58-b222-90d47dc3eb34" containerName="payload-processing" containerID="cri-o://e3e05254ae57ce540d1a3ab50f3db3b00e46b6fc2b7d13f3298dcac0a44d62d5" gracePeriod=30
Sep 11 13:50:53.339092 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.339063    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"]
Sep 11 13:50:53.339281 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.339258    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk" podUID="f2a76b58-aafd-4a9c-a999-4527253e8b9c" containerName="payload-pre-processing" containerID="cri-o://e89af5b31ff3f918dfac41f5343e8b1fa49dbbb677dace0241a0d0b4a13ca426" gracePeriod=30
Sep 11 13:50:53.615874 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.615853    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"
Sep 11 13:50:53.618876 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.618731    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"
Sep 11 13:50:53.730594 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.730564    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2a76b58-aafd-4a9c-a999-4527253e8b9c-plugins-config-volume\") pod \"f2a76b58-aafd-4a9c-a999-4527253e8b9c\" (UID: \"f2a76b58-aafd-4a9c-a999-4527253e8b9c\") "
Sep 11 13:50:53.730594 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.730599    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bfc7841-f1da-4e58-b222-90d47dc3eb34-plugins-config-volume\") pod \"6bfc7841-f1da-4e58-b222-90d47dc3eb34\" (UID: \"6bfc7841-f1da-4e58-b222-90d47dc3eb34\") "
Sep 11 13:50:53.730849 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.730672    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99qhv\" (UniqueName: \"kubernetes.io/projected/f2a76b58-aafd-4a9c-a999-4527253e8b9c-kube-api-access-99qhv\") pod \"f2a76b58-aafd-4a9c-a999-4527253e8b9c\" (UID: \"f2a76b58-aafd-4a9c-a999-4527253e8b9c\") "
Sep 11 13:50:53.730849 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.730767    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h68ff\" (UniqueName: \"kubernetes.io/projected/6bfc7841-f1da-4e58-b222-90d47dc3eb34-kube-api-access-h68ff\") pod \"6bfc7841-f1da-4e58-b222-90d47dc3eb34\" (UID: \"6bfc7841-f1da-4e58-b222-90d47dc3eb34\") "
Sep 11 13:50:53.731047 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.731024    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bfc7841-f1da-4e58-b222-90d47dc3eb34-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6bfc7841-f1da-4e58-b222-90d47dc3eb34" (UID: "6bfc7841-f1da-4e58-b222-90d47dc3eb34"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:53.731107 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.731052    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2a76b58-aafd-4a9c-a999-4527253e8b9c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f2a76b58-aafd-4a9c-a999-4527253e8b9c" (UID: "f2a76b58-aafd-4a9c-a999-4527253e8b9c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:53.732754 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.732725    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2a76b58-aafd-4a9c-a999-4527253e8b9c-kube-api-access-99qhv" (OuterVolumeSpecName: "kube-api-access-99qhv") pod "f2a76b58-aafd-4a9c-a999-4527253e8b9c" (UID: "f2a76b58-aafd-4a9c-a999-4527253e8b9c"). InnerVolumeSpecName "kube-api-access-99qhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:53.732947 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.732917    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bfc7841-f1da-4e58-b222-90d47dc3eb34-kube-api-access-h68ff" (OuterVolumeSpecName: "kube-api-access-h68ff") pod "6bfc7841-f1da-4e58-b222-90d47dc3eb34" (UID: "6bfc7841-f1da-4e58-b222-90d47dc3eb34"). InnerVolumeSpecName "kube-api-access-h68ff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:53.832346 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.832314    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h68ff\" (UniqueName: \"kubernetes.io/projected/6bfc7841-f1da-4e58-b222-90d47dc3eb34-kube-api-access-h68ff\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:53.832346 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.832342    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2a76b58-aafd-4a9c-a999-4527253e8b9c-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:53.832346 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.832350    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bfc7841-f1da-4e58-b222-90d47dc3eb34-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:53.832584 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:53.832359    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99qhv\" (UniqueName: \"kubernetes.io/projected/f2a76b58-aafd-4a9c-a999-4527253e8b9c-kube-api-access-99qhv\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:54.316127 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.316093    2580 generic.go:356] "Generic (PLEG): container finished" podID="6bfc7841-f1da-4e58-b222-90d47dc3eb34" containerID="e3e05254ae57ce540d1a3ab50f3db3b00e46b6fc2b7d13f3298dcac0a44d62d5" exitCode=0
Sep 11 13:50:54.316127 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.316127    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"
Sep 11 13:50:54.316401 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.316174    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx" event={"ID":"6bfc7841-f1da-4e58-b222-90d47dc3eb34","Type":"ContainerDied","Data":"e3e05254ae57ce540d1a3ab50f3db3b00e46b6fc2b7d13f3298dcac0a44d62d5"}
Sep 11 13:50:54.316401 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.316228    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx" event={"ID":"6bfc7841-f1da-4e58-b222-90d47dc3eb34","Type":"ContainerDied","Data":"0ec66a43a3f9875c9fee8394a001cc325d7cd138ea6da028a9c1f83fc62d28f7"}
Sep 11 13:50:54.316401 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.316257    2580 scope.go:117] "RemoveContainer" containerID="e3e05254ae57ce540d1a3ab50f3db3b00e46b6fc2b7d13f3298dcac0a44d62d5"
Sep 11 13:50:54.317619 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.317593    2580 generic.go:356] "Generic (PLEG): container finished" podID="f2a76b58-aafd-4a9c-a999-4527253e8b9c" containerID="e89af5b31ff3f918dfac41f5343e8b1fa49dbbb677dace0241a0d0b4a13ca426" exitCode=0
Sep 11 13:50:54.317727 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.317629    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"
Sep 11 13:50:54.317727 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.317677    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk" event={"ID":"f2a76b58-aafd-4a9c-a999-4527253e8b9c","Type":"ContainerDied","Data":"e89af5b31ff3f918dfac41f5343e8b1fa49dbbb677dace0241a0d0b4a13ca426"}
Sep 11 13:50:54.317816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.317733    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk" event={"ID":"f2a76b58-aafd-4a9c-a999-4527253e8b9c","Type":"ContainerDied","Data":"4400d5ed82d9e3684f6798960b19598a185bfce21cea9092499ddbb640ba7e18"}
Sep 11 13:50:54.327163 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.327145    2580 scope.go:117] "RemoveContainer" containerID="e3e05254ae57ce540d1a3ab50f3db3b00e46b6fc2b7d13f3298dcac0a44d62d5"
Sep 11 13:50:54.327467 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:50:54.327450    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3e05254ae57ce540d1a3ab50f3db3b00e46b6fc2b7d13f3298dcac0a44d62d5\": container with ID starting with e3e05254ae57ce540d1a3ab50f3db3b00e46b6fc2b7d13f3298dcac0a44d62d5 not found: ID does not exist" containerID="e3e05254ae57ce540d1a3ab50f3db3b00e46b6fc2b7d13f3298dcac0a44d62d5"
Sep 11 13:50:54.327513 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.327477    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3e05254ae57ce540d1a3ab50f3db3b00e46b6fc2b7d13f3298dcac0a44d62d5"} err="failed to get container status \"e3e05254ae57ce540d1a3ab50f3db3b00e46b6fc2b7d13f3298dcac0a44d62d5\": rpc error: code = NotFound desc = could not find container \"e3e05254ae57ce540d1a3ab50f3db3b00e46b6fc2b7d13f3298dcac0a44d62d5\": container with ID starting with e3e05254ae57ce540d1a3ab50f3db3b00e46b6fc2b7d13f3298dcac0a44d62d5 not found: ID does not exist"
Sep 11 13:50:54.327513 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.327493    2580 scope.go:117] "RemoveContainer" containerID="e89af5b31ff3f918dfac41f5343e8b1fa49dbbb677dace0241a0d0b4a13ca426"
Sep 11 13:50:54.336704 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.336516    2580 scope.go:117] "RemoveContainer" containerID="e89af5b31ff3f918dfac41f5343e8b1fa49dbbb677dace0241a0d0b4a13ca426"
Sep 11 13:50:54.336959 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:50:54.336813    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e89af5b31ff3f918dfac41f5343e8b1fa49dbbb677dace0241a0d0b4a13ca426\": container with ID starting with e89af5b31ff3f918dfac41f5343e8b1fa49dbbb677dace0241a0d0b4a13ca426 not found: ID does not exist" containerID="e89af5b31ff3f918dfac41f5343e8b1fa49dbbb677dace0241a0d0b4a13ca426"
Sep 11 13:50:54.336959 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.336835    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e89af5b31ff3f918dfac41f5343e8b1fa49dbbb677dace0241a0d0b4a13ca426"} err="failed to get container status \"e89af5b31ff3f918dfac41f5343e8b1fa49dbbb677dace0241a0d0b4a13ca426\": rpc error: code = NotFound desc = could not find container \"e89af5b31ff3f918dfac41f5343e8b1fa49dbbb677dace0241a0d0b4a13ca426\": container with ID starting with e89af5b31ff3f918dfac41f5343e8b1fa49dbbb677dace0241a0d0b4a13ca426 not found: ID does not exist"
Sep 11 13:50:54.344183 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.344162    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"]
Sep 11 13:50:54.349355 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.349333    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-082f62-6d44bd5c74-lgtvk"]
Sep 11 13:50:54.360007 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.359985    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"]
Sep 11 13:50:54.362424 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:54.362402    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-082f62-86787fff6c-9kbnx"]
Sep 11 13:50:55.317151 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:55.317115    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bfc7841-f1da-4e58-b222-90d47dc3eb34" path="/var/lib/kubelet/pods/6bfc7841-f1da-4e58-b222-90d47dc3eb34/volumes"
Sep 11 13:50:55.317512 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:55.317499    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2a76b58-aafd-4a9c-a999-4527253e8b9c" path="/var/lib/kubelet/pods/f2a76b58-aafd-4a9c-a999-4527253e8b9c/volumes"
Sep 11 13:50:55.918195 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:55.918150    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq" podUID="2bb69ce4-5d77-4d0c-abfe-517da5af7bf5" containerName="main" containerID="cri-o://f0bbe182d11fd402363556b2e6ad54e36dfd84e7f764e766ab4253ddd3aa0379" gracePeriod=45
Sep 11 13:50:55.924338 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:55.924305    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv" podUID="3c9ce5ed-23fe-4712-8dfb-09f1179cdfec" containerName="main" containerID="cri-o://4b6fdfa704c79b823e4e31d69b209c728c7ceb408bc8e6a96d6074eb21d5be16" gracePeriod=45
Sep 11 13:50:55.937674 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:55.937640    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr" podUID="4c57f759-a7ac-457d-8233-6bc2be4dc31d" containerName="main" containerID="cri-o://b4e6bebf805e58a2f64ebe31c79e8d22ce1864d5878aad949a883b475c98b8df" gracePeriod=45
Sep 11 13:50:55.956835 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:55.956801    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw" podUID="b4a4416d-224d-4415-a78c-047bb76e3fef" containerName="main" containerID="cri-o://e5a6cf462e5c9632e54cc4e74f040cf4ee975d3e81ae7db42e823ae2c9b6a096" gracePeriod=45
Sep 11 13:50:55.967256 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:55.967229    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb" podUID="139c5206-1ca9-4779-b2b0-4c34717323e0" containerName="main" containerID="cri-o://05314b9fbd6f74d8679d775148e5228825b29e4a20f80649516b59d4b659e4b7" gracePeriod=45
Sep 11 13:50:56.250040 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.250013    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:50:56.255096 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.255072    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:50:56.313285 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.313260    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:50:56.317548 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.317525    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:50:56.321261 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.321244    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:50:56.336982 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.336944    2580 generic.go:356] "Generic (PLEG): container finished" podID="4c57f759-a7ac-457d-8233-6bc2be4dc31d" containerID="b4e6bebf805e58a2f64ebe31c79e8d22ce1864d5878aad949a883b475c98b8df" exitCode=0
Sep 11 13:50:56.337168 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.337060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr" event={"ID":"4c57f759-a7ac-457d-8233-6bc2be4dc31d","Type":"ContainerDied","Data":"b4e6bebf805e58a2f64ebe31c79e8d22ce1864d5878aad949a883b475c98b8df"}
Sep 11 13:50:56.337243 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.337202    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr" event={"ID":"4c57f759-a7ac-457d-8233-6bc2be4dc31d","Type":"ContainerDied","Data":"fe652245a350de6e370dc1b8dea0d25bc7a958e380705a5c6c00a439334e74ae"}
Sep 11 13:50:56.337303 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.337077    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"
Sep 11 13:50:56.337303 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.337262    2580 scope.go:117] "RemoveContainer" containerID="b4e6bebf805e58a2f64ebe31c79e8d22ce1864d5878aad949a883b475c98b8df"
Sep 11 13:50:56.339596 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.339559    2580 generic.go:356] "Generic (PLEG): container finished" podID="b4a4416d-224d-4415-a78c-047bb76e3fef" containerID="e5a6cf462e5c9632e54cc4e74f040cf4ee975d3e81ae7db42e823ae2c9b6a096" exitCode=0
Sep 11 13:50:56.339731 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.339677    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"
Sep 11 13:50:56.339837 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.339803    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw" event={"ID":"b4a4416d-224d-4415-a78c-047bb76e3fef","Type":"ContainerDied","Data":"e5a6cf462e5c9632e54cc4e74f040cf4ee975d3e81ae7db42e823ae2c9b6a096"}
Sep 11 13:50:56.339952 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.339837    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw" event={"ID":"b4a4416d-224d-4415-a78c-047bb76e3fef","Type":"ContainerDied","Data":"5fc5886990b2f246a9109db67a9aa62a8fc11cf939eeac69d5deefd46b818f55"}
Sep 11 13:50:56.341689 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.341664    2580 generic.go:356] "Generic (PLEG): container finished" podID="3c9ce5ed-23fe-4712-8dfb-09f1179cdfec" containerID="4b6fdfa704c79b823e4e31d69b209c728c7ceb408bc8e6a96d6074eb21d5be16" exitCode=0
Sep 11 13:50:56.341806 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.341769    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv" event={"ID":"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec","Type":"ContainerDied","Data":"4b6fdfa704c79b823e4e31d69b209c728c7ceb408bc8e6a96d6074eb21d5be16"}
Sep 11 13:50:56.341884 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.341822    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv" event={"ID":"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec","Type":"ContainerDied","Data":"1e4b029fa050fa10b7847a56fcb4035d9a440a6d920be82542d0a2116038b196"}
Sep 11 13:50:56.342044 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.342019    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"
Sep 11 13:50:56.343702 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.343678    2580 generic.go:356] "Generic (PLEG): container finished" podID="139c5206-1ca9-4779-b2b0-4c34717323e0" containerID="05314b9fbd6f74d8679d775148e5228825b29e4a20f80649516b59d4b659e4b7" exitCode=0
Sep 11 13:50:56.343824 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.343786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb" event={"ID":"139c5206-1ca9-4779-b2b0-4c34717323e0","Type":"ContainerDied","Data":"05314b9fbd6f74d8679d775148e5228825b29e4a20f80649516b59d4b659e4b7"}
Sep 11 13:50:56.343824 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.343817    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb" event={"ID":"139c5206-1ca9-4779-b2b0-4c34717323e0","Type":"ContainerDied","Data":"6fd4fa4306224ff9b067225e9b82abd2dcd673c3662dcd61f916bf459e95ef53"}
Sep 11 13:50:56.344016 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.344000    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"
Sep 11 13:50:56.345259 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.345241    2580 generic.go:356] "Generic (PLEG): container finished" podID="2bb69ce4-5d77-4d0c-abfe-517da5af7bf5" containerID="f0bbe182d11fd402363556b2e6ad54e36dfd84e7f764e766ab4253ddd3aa0379" exitCode=0
Sep 11 13:50:56.345366 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.345264    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq" event={"ID":"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5","Type":"ContainerDied","Data":"f0bbe182d11fd402363556b2e6ad54e36dfd84e7f764e766ab4253ddd3aa0379"}
Sep 11 13:50:56.345366 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.345302    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq" event={"ID":"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5","Type":"ContainerDied","Data":"6b9a5525555e1f7649a117bc357d335e641c11c1010af87223ad42084cc63871"}
Sep 11 13:50:56.345366 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.345277    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"
Sep 11 13:50:56.349179 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.349157    2580 scope.go:117] "RemoveContainer" containerID="b4e6bebf805e58a2f64ebe31c79e8d22ce1864d5878aad949a883b475c98b8df"
Sep 11 13:50:56.349449 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:50:56.349431    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4e6bebf805e58a2f64ebe31c79e8d22ce1864d5878aad949a883b475c98b8df\": container with ID starting with b4e6bebf805e58a2f64ebe31c79e8d22ce1864d5878aad949a883b475c98b8df not found: ID does not exist" containerID="b4e6bebf805e58a2f64ebe31c79e8d22ce1864d5878aad949a883b475c98b8df"
Sep 11 13:50:56.349528 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.349459    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4e6bebf805e58a2f64ebe31c79e8d22ce1864d5878aad949a883b475c98b8df"} err="failed to get container status \"b4e6bebf805e58a2f64ebe31c79e8d22ce1864d5878aad949a883b475c98b8df\": rpc error: code = NotFound desc = could not find container \"b4e6bebf805e58a2f64ebe31c79e8d22ce1864d5878aad949a883b475c98b8df\": container with ID starting with b4e6bebf805e58a2f64ebe31c79e8d22ce1864d5878aad949a883b475c98b8df not found: ID does not exist"
Sep 11 13:50:56.349528 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.349475    2580 scope.go:117] "RemoveContainer" containerID="e5a6cf462e5c9632e54cc4e74f040cf4ee975d3e81ae7db42e823ae2c9b6a096"
Sep 11 13:50:56.357820 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.357796    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-tmp-dir\") pod \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") "
Sep 11 13:50:56.359375 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.359355    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-home\") pod \"b4a4416d-224d-4415-a78c-047bb76e3fef\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") "
Sep 11 13:50:56.359617 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.359602    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-dshm\") pod \"b4a4416d-224d-4415-a78c-047bb76e3fef\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") "
Sep 11 13:50:56.359745 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.359732    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-tmp-dir\") pod \"b4a4416d-224d-4415-a78c-047bb76e3fef\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") "
Sep 11 13:50:56.359859 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.359845    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-model-cache\") pod \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") "
Sep 11 13:50:56.361479 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.361459    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5gf8\" (UniqueName: \"kubernetes.io/projected/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-kube-api-access-s5gf8\") pod \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") "
Sep 11 13:50:56.361877 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.361859    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsp49\" (UniqueName: \"kubernetes.io/projected/b4a4416d-224d-4415-a78c-047bb76e3fef-kube-api-access-gsp49\") pod \"b4a4416d-224d-4415-a78c-047bb76e3fef\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") "
Sep 11 13:50:56.362034 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.362019    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-model-cache\") pod \"b4a4416d-224d-4415-a78c-047bb76e3fef\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") "
Sep 11 13:50:56.362142 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.362116    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b4a4416d-224d-4415-a78c-047bb76e3fef" (UID: "b4a4416d-224d-4415-a78c-047bb76e3fef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.362245 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.362229    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-tls-certs\") pod \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") "
Sep 11 13:50:56.362379 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.362364    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4a4416d-224d-4415-a78c-047bb76e3fef-tls-certs\") pod \"b4a4416d-224d-4415-a78c-047bb76e3fef\" (UID: \"b4a4416d-224d-4415-a78c-047bb76e3fef\") "
Sep 11 13:50:56.362481 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.362468    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-dshm\") pod \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") "
Sep 11 13:50:56.362625 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.362610    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-home\") pod \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\" (UID: \"2bb69ce4-5d77-4d0c-abfe-517da5af7bf5\") "
Sep 11 13:50:56.362827 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.362270    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-home" (OuterVolumeSpecName: "home") pod "b4a4416d-224d-4415-a78c-047bb76e3fef" (UID: "b4a4416d-224d-4415-a78c-047bb76e3fef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.363055 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.362338    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-model-cache" (OuterVolumeSpecName: "model-cache") pod "2bb69ce4-5d77-4d0c-abfe-517da5af7bf5" (UID: "2bb69ce4-5d77-4d0c-abfe-517da5af7bf5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.363055 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.362469    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2bb69ce4-5d77-4d0c-abfe-517da5af7bf5" (UID: "2bb69ce4-5d77-4d0c-abfe-517da5af7bf5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.363055 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.362768    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-model-cache" (OuterVolumeSpecName: "model-cache") pod "b4a4416d-224d-4415-a78c-047bb76e3fef" (UID: "b4a4416d-224d-4415-a78c-047bb76e3fef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.363256 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.363185    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.363256 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.363206    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.363256 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.363219    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.363256 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.363231    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.363256 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.363242    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.363738 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.363717    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-home" (OuterVolumeSpecName: "home") pod "2bb69ce4-5d77-4d0c-abfe-517da5af7bf5" (UID: "2bb69ce4-5d77-4d0c-abfe-517da5af7bf5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.365068 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.365044    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-kube-api-access-s5gf8" (OuterVolumeSpecName: "kube-api-access-s5gf8") pod "2bb69ce4-5d77-4d0c-abfe-517da5af7bf5" (UID: "2bb69ce4-5d77-4d0c-abfe-517da5af7bf5"). InnerVolumeSpecName "kube-api-access-s5gf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:56.365636 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.365611    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-dshm" (OuterVolumeSpecName: "dshm") pod "b4a4416d-224d-4415-a78c-047bb76e3fef" (UID: "b4a4416d-224d-4415-a78c-047bb76e3fef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.365836 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.365800    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4a4416d-224d-4415-a78c-047bb76e3fef-kube-api-access-gsp49" (OuterVolumeSpecName: "kube-api-access-gsp49") pod "b4a4416d-224d-4415-a78c-047bb76e3fef" (UID: "b4a4416d-224d-4415-a78c-047bb76e3fef"). InnerVolumeSpecName "kube-api-access-gsp49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:56.366223 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.366198    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4a4416d-224d-4415-a78c-047bb76e3fef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b4a4416d-224d-4415-a78c-047bb76e3fef" (UID: "b4a4416d-224d-4415-a78c-047bb76e3fef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:50:56.367108 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.367073    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-dshm" (OuterVolumeSpecName: "dshm") pod "2bb69ce4-5d77-4d0c-abfe-517da5af7bf5" (UID: "2bb69ce4-5d77-4d0c-abfe-517da5af7bf5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.368013 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.367986    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2bb69ce4-5d77-4d0c-abfe-517da5af7bf5" (UID: "2bb69ce4-5d77-4d0c-abfe-517da5af7bf5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:50:56.369162 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.369134    2580 scope.go:117] "RemoveContainer" containerID="e5a6cf462e5c9632e54cc4e74f040cf4ee975d3e81ae7db42e823ae2c9b6a096"
Sep 11 13:50:56.369515 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:50:56.369494    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5a6cf462e5c9632e54cc4e74f040cf4ee975d3e81ae7db42e823ae2c9b6a096\": container with ID starting with e5a6cf462e5c9632e54cc4e74f040cf4ee975d3e81ae7db42e823ae2c9b6a096 not found: ID does not exist" containerID="e5a6cf462e5c9632e54cc4e74f040cf4ee975d3e81ae7db42e823ae2c9b6a096"
Sep 11 13:50:56.369601 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.369526    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5a6cf462e5c9632e54cc4e74f040cf4ee975d3e81ae7db42e823ae2c9b6a096"} err="failed to get container status \"e5a6cf462e5c9632e54cc4e74f040cf4ee975d3e81ae7db42e823ae2c9b6a096\": rpc error: code = NotFound desc = could not find container \"e5a6cf462e5c9632e54cc4e74f040cf4ee975d3e81ae7db42e823ae2c9b6a096\": container with ID starting with e5a6cf462e5c9632e54cc4e74f040cf4ee975d3e81ae7db42e823ae2c9b6a096 not found: ID does not exist"
Sep 11 13:50:56.369601 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.369552    2580 scope.go:117] "RemoveContainer" containerID="4b6fdfa704c79b823e4e31d69b209c728c7ceb408bc8e6a96d6074eb21d5be16"
Sep 11 13:50:56.384719 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.384694    2580 scope.go:117] "RemoveContainer" containerID="4b6fdfa704c79b823e4e31d69b209c728c7ceb408bc8e6a96d6074eb21d5be16"
Sep 11 13:50:56.385132 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:50:56.385109    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b6fdfa704c79b823e4e31d69b209c728c7ceb408bc8e6a96d6074eb21d5be16\": container with ID starting with 4b6fdfa704c79b823e4e31d69b209c728c7ceb408bc8e6a96d6074eb21d5be16 not found: ID does not exist" containerID="4b6fdfa704c79b823e4e31d69b209c728c7ceb408bc8e6a96d6074eb21d5be16"
Sep 11 13:50:56.385234 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.385138    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6fdfa704c79b823e4e31d69b209c728c7ceb408bc8e6a96d6074eb21d5be16"} err="failed to get container status \"4b6fdfa704c79b823e4e31d69b209c728c7ceb408bc8e6a96d6074eb21d5be16\": rpc error: code = NotFound desc = could not find container \"4b6fdfa704c79b823e4e31d69b209c728c7ceb408bc8e6a96d6074eb21d5be16\": container with ID starting with 4b6fdfa704c79b823e4e31d69b209c728c7ceb408bc8e6a96d6074eb21d5be16 not found: ID does not exist"
Sep 11 13:50:56.385234 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.385158    2580 scope.go:117] "RemoveContainer" containerID="05314b9fbd6f74d8679d775148e5228825b29e4a20f80649516b59d4b659e4b7"
Sep 11 13:50:56.395833 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.395813    2580 scope.go:117] "RemoveContainer" containerID="05314b9fbd6f74d8679d775148e5228825b29e4a20f80649516b59d4b659e4b7"
Sep 11 13:50:56.396205 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:50:56.396180    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05314b9fbd6f74d8679d775148e5228825b29e4a20f80649516b59d4b659e4b7\": container with ID starting with 05314b9fbd6f74d8679d775148e5228825b29e4a20f80649516b59d4b659e4b7 not found: ID does not exist" containerID="05314b9fbd6f74d8679d775148e5228825b29e4a20f80649516b59d4b659e4b7"
Sep 11 13:50:56.396316 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.396216    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05314b9fbd6f74d8679d775148e5228825b29e4a20f80649516b59d4b659e4b7"} err="failed to get container status \"05314b9fbd6f74d8679d775148e5228825b29e4a20f80649516b59d4b659e4b7\": rpc error: code = NotFound desc = could not find container \"05314b9fbd6f74d8679d775148e5228825b29e4a20f80649516b59d4b659e4b7\": container with ID starting with 05314b9fbd6f74d8679d775148e5228825b29e4a20f80649516b59d4b659e4b7 not found: ID does not exist"
Sep 11 13:50:56.396316 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.396241    2580 scope.go:117] "RemoveContainer" containerID="f0bbe182d11fd402363556b2e6ad54e36dfd84e7f764e766ab4253ddd3aa0379"
Sep 11 13:50:56.407187 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.407166    2580 scope.go:117] "RemoveContainer" containerID="f0bbe182d11fd402363556b2e6ad54e36dfd84e7f764e766ab4253ddd3aa0379"
Sep 11 13:50:56.407506 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:50:56.407489    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0bbe182d11fd402363556b2e6ad54e36dfd84e7f764e766ab4253ddd3aa0379\": container with ID starting with f0bbe182d11fd402363556b2e6ad54e36dfd84e7f764e766ab4253ddd3aa0379 not found: ID does not exist" containerID="f0bbe182d11fd402363556b2e6ad54e36dfd84e7f764e766ab4253ddd3aa0379"
Sep 11 13:50:56.407576 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.407518    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0bbe182d11fd402363556b2e6ad54e36dfd84e7f764e766ab4253ddd3aa0379"} err="failed to get container status \"f0bbe182d11fd402363556b2e6ad54e36dfd84e7f764e766ab4253ddd3aa0379\": rpc error: code = NotFound desc = could not find container \"f0bbe182d11fd402363556b2e6ad54e36dfd84e7f764e766ab4253ddd3aa0379\": container with ID starting with f0bbe182d11fd402363556b2e6ad54e36dfd84e7f764e766ab4253ddd3aa0379 not found: ID does not exist"
Sep 11 13:50:56.464578 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.464482    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-model-cache\") pod \"139c5206-1ca9-4779-b2b0-4c34717323e0\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") "
Sep 11 13:50:56.464578 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.464522    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-model-cache\") pod \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") "
Sep 11 13:50:56.464578 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.464563    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-home\") pod \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") "
Sep 11 13:50:56.464843 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.464593    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-dshm\") pod \"139c5206-1ca9-4779-b2b0-4c34717323e0\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") "
Sep 11 13:50:56.464843 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.464624    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c57f759-a7ac-457d-8233-6bc2be4dc31d-tls-certs\") pod \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") "
Sep 11 13:50:56.464843 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.464675    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/139c5206-1ca9-4779-b2b0-4c34717323e0-tls-certs\") pod \"139c5206-1ca9-4779-b2b0-4c34717323e0\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") "
Sep 11 13:50:56.464843 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.464700    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-tmp-dir\") pod \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") "
Sep 11 13:50:56.464843 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.464806    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrtsl\" (UniqueName: \"kubernetes.io/projected/139c5206-1ca9-4779-b2b0-4c34717323e0-kube-api-access-wrtsl\") pod \"139c5206-1ca9-4779-b2b0-4c34717323e0\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") "
Sep 11 13:50:56.465147 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.464805    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-model-cache" (OuterVolumeSpecName: "model-cache") pod "139c5206-1ca9-4779-b2b0-4c34717323e0" (UID: "139c5206-1ca9-4779-b2b0-4c34717323e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.465147 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.464809    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-model-cache" (OuterVolumeSpecName: "model-cache") pod "3c9ce5ed-23fe-4712-8dfb-09f1179cdfec" (UID: "3c9ce5ed-23fe-4712-8dfb-09f1179cdfec"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.465147 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.464837    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-home" (OuterVolumeSpecName: "home") pod "3c9ce5ed-23fe-4712-8dfb-09f1179cdfec" (UID: "3c9ce5ed-23fe-4712-8dfb-09f1179cdfec"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.465147 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.464843    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrkff\" (UniqueName: \"kubernetes.io/projected/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-kube-api-access-rrkff\") pod \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") "
Sep 11 13:50:56.465147 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.464967    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-dshm\") pod \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") "
Sep 11 13:50:56.465147 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.464986    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4c57f759-a7ac-457d-8233-6bc2be4dc31d" (UID: "4c57f759-a7ac-457d-8233-6bc2be4dc31d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.465147 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.464997    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t67px\" (UniqueName: \"kubernetes.io/projected/4c57f759-a7ac-457d-8233-6bc2be4dc31d-kube-api-access-t67px\") pod \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") "
Sep 11 13:50:56.465147 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.465052    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-home\") pod \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") "
Sep 11 13:50:56.465147 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.465080    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-model-cache\") pod \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\" (UID: \"4c57f759-a7ac-457d-8233-6bc2be4dc31d\") "
Sep 11 13:50:56.465147 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.465114    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-tmp-dir\") pod \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") "
Sep 11 13:50:56.465147 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.465152    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-tmp-dir\") pod \"139c5206-1ca9-4779-b2b0-4c34717323e0\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") "
Sep 11 13:50:56.465670 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.465193    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-tls-certs\") pod \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") "
Sep 11 13:50:56.465670 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.465248    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-home\") pod \"139c5206-1ca9-4779-b2b0-4c34717323e0\" (UID: \"139c5206-1ca9-4779-b2b0-4c34717323e0\") "
Sep 11 13:50:56.465670 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.465313    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-dshm\") pod \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\" (UID: \"3c9ce5ed-23fe-4712-8dfb-09f1179cdfec\") "
Sep 11 13:50:56.465670 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.465479    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3c9ce5ed-23fe-4712-8dfb-09f1179cdfec" (UID: "3c9ce5ed-23fe-4712-8dfb-09f1179cdfec"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.465886 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.465820    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-home" (OuterVolumeSpecName: "home") pod "4c57f759-a7ac-457d-8233-6bc2be4dc31d" (UID: "4c57f759-a7ac-457d-8233-6bc2be4dc31d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.466048 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.466026    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "139c5206-1ca9-4779-b2b0-4c34717323e0" (UID: "139c5206-1ca9-4779-b2b0-4c34717323e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.466133 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.466116    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.466271 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.466228    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-model-cache" (OuterVolumeSpecName: "model-cache") pod "4c57f759-a7ac-457d-8233-6bc2be4dc31d" (UID: "4c57f759-a7ac-457d-8233-6bc2be4dc31d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.466394 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.466307    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.466394 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.466328    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4a4416d-224d-4415-a78c-047bb76e3fef-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.466394 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.466342    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.466394 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.466355    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.466394 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.466371    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5gf8\" (UniqueName: \"kubernetes.io/projected/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-kube-api-access-s5gf8\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.466394 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.466384    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.466750 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.466399    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsp49\" (UniqueName: \"kubernetes.io/projected/b4a4416d-224d-4415-a78c-047bb76e3fef-kube-api-access-gsp49\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.466750 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.466414    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.466750 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.466429    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4a4416d-224d-4415-a78c-047bb76e3fef-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.466750 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.466442    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.466750 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.466455    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.467062 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.466805    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-home" (OuterVolumeSpecName: "home") pod "139c5206-1ca9-4779-b2b0-4c34717323e0" (UID: "139c5206-1ca9-4779-b2b0-4c34717323e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.467227 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.467203    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-dshm" (OuterVolumeSpecName: "dshm") pod "139c5206-1ca9-4779-b2b0-4c34717323e0" (UID: "139c5206-1ca9-4779-b2b0-4c34717323e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.467667 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.467606    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-dshm" (OuterVolumeSpecName: "dshm") pod "4c57f759-a7ac-457d-8233-6bc2be4dc31d" (UID: "4c57f759-a7ac-457d-8233-6bc2be4dc31d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.467773 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.467699    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c57f759-a7ac-457d-8233-6bc2be4dc31d-kube-api-access-t67px" (OuterVolumeSpecName: "kube-api-access-t67px") pod "4c57f759-a7ac-457d-8233-6bc2be4dc31d" (UID: "4c57f759-a7ac-457d-8233-6bc2be4dc31d"). InnerVolumeSpecName "kube-api-access-t67px". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:56.468065 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.468042    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c57f759-a7ac-457d-8233-6bc2be4dc31d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4c57f759-a7ac-457d-8233-6bc2be4dc31d" (UID: "4c57f759-a7ac-457d-8233-6bc2be4dc31d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:50:56.468158 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.468112    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/139c5206-1ca9-4779-b2b0-4c34717323e0-kube-api-access-wrtsl" (OuterVolumeSpecName: "kube-api-access-wrtsl") pod "139c5206-1ca9-4779-b2b0-4c34717323e0" (UID: "139c5206-1ca9-4779-b2b0-4c34717323e0"). InnerVolumeSpecName "kube-api-access-wrtsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:56.468295 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.468278    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3c9ce5ed-23fe-4712-8dfb-09f1179cdfec" (UID: "3c9ce5ed-23fe-4712-8dfb-09f1179cdfec"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:50:56.468584 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.468567    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-kube-api-access-rrkff" (OuterVolumeSpecName: "kube-api-access-rrkff") pod "3c9ce5ed-23fe-4712-8dfb-09f1179cdfec" (UID: "3c9ce5ed-23fe-4712-8dfb-09f1179cdfec"). InnerVolumeSpecName "kube-api-access-rrkff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:56.469063 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.469040    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/139c5206-1ca9-4779-b2b0-4c34717323e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "139c5206-1ca9-4779-b2b0-4c34717323e0" (UID: "139c5206-1ca9-4779-b2b0-4c34717323e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:50:56.469156 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.469141    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-dshm" (OuterVolumeSpecName: "dshm") pod "3c9ce5ed-23fe-4712-8dfb-09f1179cdfec" (UID: "3c9ce5ed-23fe-4712-8dfb-09f1179cdfec"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:56.567357 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.567318    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.567357 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.567349    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c57f759-a7ac-457d-8233-6bc2be4dc31d-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.567357 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.567358    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/139c5206-1ca9-4779-b2b0-4c34717323e0-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.567357 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.567366    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrtsl\" (UniqueName: \"kubernetes.io/projected/139c5206-1ca9-4779-b2b0-4c34717323e0-kube-api-access-wrtsl\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.567618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.567375    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrkff\" (UniqueName: \"kubernetes.io/projected/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-kube-api-access-rrkff\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.567618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.567383    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.567618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.567394    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t67px\" (UniqueName: \"kubernetes.io/projected/4c57f759-a7ac-457d-8233-6bc2be4dc31d-kube-api-access-t67px\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.567618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.567402    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.567618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.567411    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c57f759-a7ac-457d-8233-6bc2be4dc31d-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.567618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.567418    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.567618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.567426    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.567618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.567432    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/139c5206-1ca9-4779-b2b0-4c34717323e0-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.567618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.567441    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:50:56.665464 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.665436    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"]
Sep 11 13:50:56.668449 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.668424    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/embedding-w0-082f62-kserve-59457d844-l9nbr"]
Sep 11 13:50:56.679097 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.679074    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"]
Sep 11 13:50:56.685733 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.685704    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/distinct-w0-082f62-kserve-8bf4bfc6c-bf6pv"]
Sep 11 13:50:56.696641 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.696620    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"]
Sep 11 13:50:56.700894 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.700869    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/simulator-w0-082f62-kserve-dd7b776b9-7xfmb"]
Sep 11 13:50:56.710082 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.710060    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"]
Sep 11 13:50:56.713988 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.713963    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/premium-w0-082f62-kserve-7f466dd67b-sphgw"]
Sep 11 13:50:56.724702 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.724645    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"]
Sep 11 13:50:56.729055 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:56.729036    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-082f62/distinct-2-w0-082f62-kserve-5665b9cf6-5brnq"]
Sep 11 13:50:57.318221 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:57.318186    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="139c5206-1ca9-4779-b2b0-4c34717323e0" path="/var/lib/kubelet/pods/139c5206-1ca9-4779-b2b0-4c34717323e0/volumes"
Sep 11 13:50:57.318774 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:57.318754    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bb69ce4-5d77-4d0c-abfe-517da5af7bf5" path="/var/lib/kubelet/pods/2bb69ce4-5d77-4d0c-abfe-517da5af7bf5/volumes"
Sep 11 13:50:57.319346 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:57.319319    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c9ce5ed-23fe-4712-8dfb-09f1179cdfec" path="/var/lib/kubelet/pods/3c9ce5ed-23fe-4712-8dfb-09f1179cdfec/volumes"
Sep 11 13:50:57.320522 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:57.320233    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c57f759-a7ac-457d-8233-6bc2be4dc31d" path="/var/lib/kubelet/pods/4c57f759-a7ac-457d-8233-6bc2be4dc31d/volumes"
Sep 11 13:50:57.320796 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:50:57.320775    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4a4416d-224d-4415-a78c-047bb76e3fef" path="/var/lib/kubelet/pods/b4a4416d-224d-4415-a78c-047bb76e3fef/volumes"
Sep 11 13:51:21.434693 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:51:21.434658    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:51:21.444115 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:51:21.444090    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:51:21.643582 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:51:21.643551    2580 scope.go:117] "RemoveContainer" containerID="6b853b6e724046730c2342b66bbcf429088f38d0345cf30e5e87a39ebcf4e6c3"
Sep 11 13:51:21.652870 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:51:21.652850    2580 scope.go:117] "RemoveContainer" containerID="c3e6104433e578eb981f158eec6cbe6006f75ada8fd859c04e4050347dc1d35f"
Sep 11 13:51:24.256667 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:51:24.256630    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:51:47.712936 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:51:47.712890    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:51:59.372150 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:51:59.372115    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"]
Sep 11 13:51:59.395390 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:51:59.395358    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"]
Sep 11 13:52:09.789042 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:09.788999    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:52:10.562899 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:10.562866    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"]
Sep 11 13:52:10.563105 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:10.563083    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc" podUID="4c4a728a-bb72-4b9f-b670-8b57bf22afdf" containerName="payload-processing" containerID="cri-o://bcfc4813b63c3d3485fb85ff0897d5348fdf1b82767b899718f32b25d671caed" gracePeriod=30
Sep 11 13:52:10.577748 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:10.577717    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"]
Sep 11 13:52:10.797848 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:10.797811    2580 generic.go:356] "Generic (PLEG): container finished" podID="4c4a728a-bb72-4b9f-b670-8b57bf22afdf" containerID="bcfc4813b63c3d3485fb85ff0897d5348fdf1b82767b899718f32b25d671caed" exitCode=0
Sep 11 13:52:10.798240 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:10.797882    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc" event={"ID":"4c4a728a-bb72-4b9f-b670-8b57bf22afdf","Type":"ContainerDied","Data":"bcfc4813b63c3d3485fb85ff0897d5348fdf1b82767b899718f32b25d671caed"}
Sep 11 13:52:10.798299 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:10.798269    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s" podUID="d0db7b13-a95c-4c5b-a021-80e0fe9eea4f" containerName="payload-pre-processing" containerID="cri-o://0f179142949109fe5ebf5c9a41ddf336ffcc69d43d221bb5cd4c747a0dadc60a" gracePeriod=30
Sep 11 13:52:10.872241 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:10.872216    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"
Sep 11 13:52:10.931023 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:10.930990    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfs6f\" (UniqueName: \"kubernetes.io/projected/4c4a728a-bb72-4b9f-b670-8b57bf22afdf-kube-api-access-vfs6f\") pod \"4c4a728a-bb72-4b9f-b670-8b57bf22afdf\" (UID: \"4c4a728a-bb72-4b9f-b670-8b57bf22afdf\") "
Sep 11 13:52:10.931166 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:10.931140    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c4a728a-bb72-4b9f-b670-8b57bf22afdf-plugins-config-volume\") pod \"4c4a728a-bb72-4b9f-b670-8b57bf22afdf\" (UID: \"4c4a728a-bb72-4b9f-b670-8b57bf22afdf\") "
Sep 11 13:52:10.931529 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:10.931492    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c4a728a-bb72-4b9f-b670-8b57bf22afdf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4c4a728a-bb72-4b9f-b670-8b57bf22afdf" (UID: "4c4a728a-bb72-4b9f-b670-8b57bf22afdf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:52:10.933085 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:10.933060    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c4a728a-bb72-4b9f-b670-8b57bf22afdf-kube-api-access-vfs6f" (OuterVolumeSpecName: "kube-api-access-vfs6f") pod "4c4a728a-bb72-4b9f-b670-8b57bf22afdf" (UID: "4c4a728a-bb72-4b9f-b670-8b57bf22afdf"). InnerVolumeSpecName "kube-api-access-vfs6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:52:11.032536 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.032494    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c4a728a-bb72-4b9f-b670-8b57bf22afdf-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:11.032536 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.032526    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfs6f\" (UniqueName: \"kubernetes.io/projected/4c4a728a-bb72-4b9f-b670-8b57bf22afdf-kube-api-access-vfs6f\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:11.046357 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.046334    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"
Sep 11 13:52:11.133767 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.133687    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tchr4\" (UniqueName: \"kubernetes.io/projected/d0db7b13-a95c-4c5b-a021-80e0fe9eea4f-kube-api-access-tchr4\") pod \"d0db7b13-a95c-4c5b-a021-80e0fe9eea4f\" (UID: \"d0db7b13-a95c-4c5b-a021-80e0fe9eea4f\") "
Sep 11 13:52:11.133934 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.133778    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0db7b13-a95c-4c5b-a021-80e0fe9eea4f-plugins-config-volume\") pod \"d0db7b13-a95c-4c5b-a021-80e0fe9eea4f\" (UID: \"d0db7b13-a95c-4c5b-a021-80e0fe9eea4f\") "
Sep 11 13:52:11.134209 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.134185    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0db7b13-a95c-4c5b-a021-80e0fe9eea4f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d0db7b13-a95c-4c5b-a021-80e0fe9eea4f" (UID: "d0db7b13-a95c-4c5b-a021-80e0fe9eea4f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:52:11.135848 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.135829    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0db7b13-a95c-4c5b-a021-80e0fe9eea4f-kube-api-access-tchr4" (OuterVolumeSpecName: "kube-api-access-tchr4") pod "d0db7b13-a95c-4c5b-a021-80e0fe9eea4f" (UID: "d0db7b13-a95c-4c5b-a021-80e0fe9eea4f"). InnerVolumeSpecName "kube-api-access-tchr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:52:11.234634 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.234598    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tchr4\" (UniqueName: \"kubernetes.io/projected/d0db7b13-a95c-4c5b-a021-80e0fe9eea4f-kube-api-access-tchr4\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:11.234634 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.234628    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0db7b13-a95c-4c5b-a021-80e0fe9eea4f-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:11.805203 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.805166    2580 generic.go:356] "Generic (PLEG): container finished" podID="d0db7b13-a95c-4c5b-a021-80e0fe9eea4f" containerID="0f179142949109fe5ebf5c9a41ddf336ffcc69d43d221bb5cd4c747a0dadc60a" exitCode=0
Sep 11 13:52:11.805687 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.805217    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"
Sep 11 13:52:11.805687 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.805248    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s" event={"ID":"d0db7b13-a95c-4c5b-a021-80e0fe9eea4f","Type":"ContainerDied","Data":"0f179142949109fe5ebf5c9a41ddf336ffcc69d43d221bb5cd4c747a0dadc60a"}
Sep 11 13:52:11.805687 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.805300    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s" event={"ID":"d0db7b13-a95c-4c5b-a021-80e0fe9eea4f","Type":"ContainerDied","Data":"0ddc8ab6371921ea65566d417a9764500866635587c908f3742363eebfa8f113"}
Sep 11 13:52:11.805687 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.805326    2580 scope.go:117] "RemoveContainer" containerID="0f179142949109fe5ebf5c9a41ddf336ffcc69d43d221bb5cd4c747a0dadc60a"
Sep 11 13:52:11.806880 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.806860    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc" event={"ID":"4c4a728a-bb72-4b9f-b670-8b57bf22afdf","Type":"ContainerDied","Data":"1424f34cde78b450db325ed632879d82c1b63410478a59b7a325b16bec14462d"}
Sep 11 13:52:11.806995 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.806915    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"
Sep 11 13:52:11.815496 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.815478    2580 scope.go:117] "RemoveContainer" containerID="0f179142949109fe5ebf5c9a41ddf336ffcc69d43d221bb5cd4c747a0dadc60a"
Sep 11 13:52:11.815771 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:52:11.815754    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f179142949109fe5ebf5c9a41ddf336ffcc69d43d221bb5cd4c747a0dadc60a\": container with ID starting with 0f179142949109fe5ebf5c9a41ddf336ffcc69d43d221bb5cd4c747a0dadc60a not found: ID does not exist" containerID="0f179142949109fe5ebf5c9a41ddf336ffcc69d43d221bb5cd4c747a0dadc60a"
Sep 11 13:52:11.815821 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.815781    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f179142949109fe5ebf5c9a41ddf336ffcc69d43d221bb5cd4c747a0dadc60a"} err="failed to get container status \"0f179142949109fe5ebf5c9a41ddf336ffcc69d43d221bb5cd4c747a0dadc60a\": rpc error: code = NotFound desc = could not find container \"0f179142949109fe5ebf5c9a41ddf336ffcc69d43d221bb5cd4c747a0dadc60a\": container with ID starting with 0f179142949109fe5ebf5c9a41ddf336ffcc69d43d221bb5cd4c747a0dadc60a not found: ID does not exist"
Sep 11 13:52:11.815821 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.815796    2580 scope.go:117] "RemoveContainer" containerID="bcfc4813b63c3d3485fb85ff0897d5348fdf1b82767b899718f32b25d671caed"
Sep 11 13:52:11.885277 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.885187    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"]
Sep 11 13:52:11.887654 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.887625    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-371c38-847b485f5d-z8v9s"]
Sep 11 13:52:11.935182 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.935143    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"]
Sep 11 13:52:11.938661 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:11.938637    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-371c38-5b77f544df-hb5qc"]
Sep 11 13:52:12.583787 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.583751    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7616-6dbd49"]
Sep 11 13:52:12.584155 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584141    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c4a728a-bb72-4b9f-b670-8b57bf22afdf" containerName="payload-processing"
Sep 11 13:52:12.584209 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584157    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c4a728a-bb72-4b9f-b670-8b57bf22afdf" containerName="payload-processing"
Sep 11 13:52:12.584209 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584165    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0db7b13-a95c-4c5b-a021-80e0fe9eea4f" containerName="payload-pre-processing"
Sep 11 13:52:12.584209 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584170    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0db7b13-a95c-4c5b-a021-80e0fe9eea4f" containerName="payload-pre-processing"
Sep 11 13:52:12.584209 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584177    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40658cad-e2d6-4d22-97f4-90a264ac2a8a" containerName="payload-pre-processing"
Sep 11 13:52:12.584209 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584182    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="40658cad-e2d6-4d22-97f4-90a264ac2a8a" containerName="payload-pre-processing"
Sep 11 13:52:12.584209 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584198    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c9ce5ed-23fe-4712-8dfb-09f1179cdfec" containerName="main"
Sep 11 13:52:12.584209 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584204    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c9ce5ed-23fe-4712-8dfb-09f1179cdfec" containerName="main"
Sep 11 13:52:12.584209 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584210    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bb69ce4-5d77-4d0c-abfe-517da5af7bf5" containerName="main"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584216    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bb69ce4-5d77-4d0c-abfe-517da5af7bf5" containerName="main"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584222    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="139c5206-1ca9-4779-b2b0-4c34717323e0" containerName="main"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584227    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="139c5206-1ca9-4779-b2b0-4c34717323e0" containerName="main"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584232    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4a4416d-224d-4415-a78c-047bb76e3fef" containerName="main"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584237    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4a4416d-224d-4415-a78c-047bb76e3fef" containerName="main"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584244    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2a76b58-aafd-4a9c-a999-4527253e8b9c" containerName="payload-pre-processing"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584249    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2a76b58-aafd-4a9c-a999-4527253e8b9c" containerName="payload-pre-processing"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584258    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c57f759-a7ac-457d-8233-6bc2be4dc31d" containerName="main"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584263    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c57f759-a7ac-457d-8233-6bc2be4dc31d" containerName="main"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584269    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bfc7841-f1da-4e58-b222-90d47dc3eb34" containerName="payload-processing"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584273    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bfc7841-f1da-4e58-b222-90d47dc3eb34" containerName="payload-processing"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584284    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db7221f8-224f-463e-851b-dbc29efa946d" containerName="revoke-keys"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584289    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="db7221f8-224f-463e-851b-dbc29efa946d" containerName="revoke-keys"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584345    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bb69ce4-5d77-4d0c-abfe-517da5af7bf5" containerName="main"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584353    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c4a728a-bb72-4b9f-b670-8b57bf22afdf" containerName="payload-processing"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584359    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0db7b13-a95c-4c5b-a021-80e0fe9eea4f" containerName="payload-pre-processing"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584364    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="db7221f8-224f-463e-851b-dbc29efa946d" containerName="revoke-keys"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584371    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4a4416d-224d-4415-a78c-047bb76e3fef" containerName="main"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584376    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="139c5206-1ca9-4779-b2b0-4c34717323e0" containerName="main"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584381    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c57f759-a7ac-457d-8233-6bc2be4dc31d" containerName="main"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584385    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bfc7841-f1da-4e58-b222-90d47dc3eb34" containerName="payload-processing"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584390    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="40658cad-e2d6-4d22-97f4-90a264ac2a8a" containerName="payload-pre-processing"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584396    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2a76b58-aafd-4a9c-a999-4527253e8b9c" containerName="payload-pre-processing"
Sep 11 13:52:12.584477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.584401    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c9ce5ed-23fe-4712-8dfb-09f1179cdfec" containerName="main"
Sep 11 13:52:12.588521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.588505    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-6dbd49"
Sep 11 13:52:12.599923 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.599880    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7616-6dbd49"]
Sep 11 13:52:12.603563 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.603538    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7ltrq\""
Sep 11 13:52:12.647485 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.647453    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h99sb\" (UniqueName: \"kubernetes.io/projected/811711e5-4689-4ee8-97f8-5f4862435e72-kube-api-access-h99sb\") pod \"test-curl-7616-6dbd49\" (UID: \"811711e5-4689-4ee8-97f8-5f4862435e72\") " pod="opendatahub/test-curl-7616-6dbd49"
Sep 11 13:52:12.748243 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.748198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h99sb\" (UniqueName: \"kubernetes.io/projected/811711e5-4689-4ee8-97f8-5f4862435e72-kube-api-access-h99sb\") pod \"test-curl-7616-6dbd49\" (UID: \"811711e5-4689-4ee8-97f8-5f4862435e72\") " pod="opendatahub/test-curl-7616-6dbd49"
Sep 11 13:52:12.757641 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.757617    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h99sb\" (UniqueName: \"kubernetes.io/projected/811711e5-4689-4ee8-97f8-5f4862435e72-kube-api-access-h99sb\") pod \"test-curl-7616-6dbd49\" (UID: \"811711e5-4689-4ee8-97f8-5f4862435e72\") " pod="opendatahub/test-curl-7616-6dbd49"
Sep 11 13:52:12.897411 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:12.897318    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-6dbd49"
Sep 11 13:52:13.179524 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:13.179497    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7616-6dbd49"]
Sep 11 13:52:13.181236 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:52:13.181210    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod811711e5_4689_4ee8_97f8_5f4862435e72.slice/crio-38afb382209c93c4e20da72c73ce333349bdb62e841ba96ed233b9a30a6ad417 WatchSource:0}: Error finding container 38afb382209c93c4e20da72c73ce333349bdb62e841ba96ed233b9a30a6ad417: Status 404 returned error can't find the container with id 38afb382209c93c4e20da72c73ce333349bdb62e841ba96ed233b9a30a6ad417
Sep 11 13:52:13.315626 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:13.315591    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c4a728a-bb72-4b9f-b670-8b57bf22afdf" path="/var/lib/kubelet/pods/4c4a728a-bb72-4b9f-b670-8b57bf22afdf/volumes"
Sep 11 13:52:13.315998 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:13.315985    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0db7b13-a95c-4c5b-a021-80e0fe9eea4f" path="/var/lib/kubelet/pods/d0db7b13-a95c-4c5b-a021-80e0fe9eea4f/volumes"
Sep 11 13:52:13.825119 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:13.825083    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7616-6dbd49" event={"ID":"811711e5-4689-4ee8-97f8-5f4862435e72","Type":"ContainerStarted","Data":"a64abab8a1d38e00a2f31039c0e2ed8f28c91555c8d945e5fe178dbfcc008db3"}
Sep 11 13:52:13.825318 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:13.825129    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7616-6dbd49" event={"ID":"811711e5-4689-4ee8-97f8-5f4862435e72","Type":"ContainerStarted","Data":"38afb382209c93c4e20da72c73ce333349bdb62e841ba96ed233b9a30a6ad417"}
Sep 11 13:52:13.848050 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:13.848000    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7616-6dbd49" podStartSLOduration=1.847985464 podStartE2EDuration="1.847985464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:52:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:52:13.846032719 +0000 UTC m=+1853.064570214" watchObservedRunningTime="2026-09-11 13:52:13.847985464 +0000 UTC m=+1853.066522950"
Sep 11 13:52:14.391994 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.391949    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726" podUID="734aa3c9-314b-4d19-ae75-6c6469ae244a" containerName="main" containerID="cri-o://cfd102db5934572cb8c9246d73609fa360b713fa6a991fbf3512f5076efe6114" gracePeriod=45
Sep 11 13:52:14.414541 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.414508    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf" podUID="5c82572f-6798-4d50-9dc4-124b7f0bd1d9" containerName="main" containerID="cri-o://38449096d2049d480c82fdd268cc053df4f8dc3d6e4a1da7c65729f2dc48ea42" gracePeriod=45
Sep 11 13:52:14.607840 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.607730    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7616-6dbd49"]
Sep 11 13:52:14.614706 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.614621    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7616-6dbd49"]
Sep 11 13:52:14.683602 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.683578    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:52:14.686444 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.686419    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:52:14.766846 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.766810    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-dshm\") pod \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") "
Sep 11 13:52:14.767043 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.766856    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-model-cache\") pod \"734aa3c9-314b-4d19-ae75-6c6469ae244a\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") "
Sep 11 13:52:14.767043 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.766884    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-model-cache\") pod \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") "
Sep 11 13:52:14.767043 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.766914    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-tmp-dir\") pod \"734aa3c9-314b-4d19-ae75-6c6469ae244a\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") "
Sep 11 13:52:14.767043 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.766946    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-tmp-dir\") pod \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") "
Sep 11 13:52:14.767043 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.766998    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-home\") pod \"734aa3c9-314b-4d19-ae75-6c6469ae244a\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") "
Sep 11 13:52:14.767043 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.767035    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5gv4\" (UniqueName: \"kubernetes.io/projected/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-kube-api-access-z5gv4\") pod \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") "
Sep 11 13:52:14.767396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.767066    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lz7mr\" (UniqueName: \"kubernetes.io/projected/734aa3c9-314b-4d19-ae75-6c6469ae244a-kube-api-access-lz7mr\") pod \"734aa3c9-314b-4d19-ae75-6c6469ae244a\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") "
Sep 11 13:52:14.767396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.767121    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-dshm\") pod \"734aa3c9-314b-4d19-ae75-6c6469ae244a\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") "
Sep 11 13:52:14.767396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.767150    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/734aa3c9-314b-4d19-ae75-6c6469ae244a-tls-certs\") pod \"734aa3c9-314b-4d19-ae75-6c6469ae244a\" (UID: \"734aa3c9-314b-4d19-ae75-6c6469ae244a\") "
Sep 11 13:52:14.767396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.767182    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "5c82572f-6798-4d50-9dc4-124b7f0bd1d9" (UID: "5c82572f-6798-4d50-9dc4-124b7f0bd1d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:14.767396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.767186    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-model-cache" (OuterVolumeSpecName: "model-cache") pod "734aa3c9-314b-4d19-ae75-6c6469ae244a" (UID: "734aa3c9-314b-4d19-ae75-6c6469ae244a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:14.767396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.767199    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "734aa3c9-314b-4d19-ae75-6c6469ae244a" (UID: "734aa3c9-314b-4d19-ae75-6c6469ae244a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:14.767396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.767229    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-tls-certs\") pod \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") "
Sep 11 13:52:14.767396 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.767257    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-home\") pod \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\" (UID: \"5c82572f-6798-4d50-9dc4-124b7f0bd1d9\") "
Sep 11 13:52:14.768976 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.767434    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5c82572f-6798-4d50-9dc4-124b7f0bd1d9" (UID: "5c82572f-6798-4d50-9dc4-124b7f0bd1d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:14.768976 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.767597    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-home" (OuterVolumeSpecName: "home") pod "734aa3c9-314b-4d19-ae75-6c6469ae244a" (UID: "734aa3c9-314b-4d19-ae75-6c6469ae244a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:14.768976 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.767644    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:14.768976 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.767663    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:14.768976 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.767676    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:14.768976 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.767689    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:14.768976 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.768775    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-home" (OuterVolumeSpecName: "home") pod "5c82572f-6798-4d50-9dc4-124b7f0bd1d9" (UID: "5c82572f-6798-4d50-9dc4-124b7f0bd1d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:14.769533 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.769508    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-dshm" (OuterVolumeSpecName: "dshm") pod "5c82572f-6798-4d50-9dc4-124b7f0bd1d9" (UID: "5c82572f-6798-4d50-9dc4-124b7f0bd1d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:14.770464 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.770228    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5c82572f-6798-4d50-9dc4-124b7f0bd1d9" (UID: "5c82572f-6798-4d50-9dc4-124b7f0bd1d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:52:14.770784 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.770757    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/734aa3c9-314b-4d19-ae75-6c6469ae244a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "734aa3c9-314b-4d19-ae75-6c6469ae244a" (UID: "734aa3c9-314b-4d19-ae75-6c6469ae244a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:52:14.771173 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.771148    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-dshm" (OuterVolumeSpecName: "dshm") pod "734aa3c9-314b-4d19-ae75-6c6469ae244a" (UID: "734aa3c9-314b-4d19-ae75-6c6469ae244a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:14.771275 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.771187    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/734aa3c9-314b-4d19-ae75-6c6469ae244a-kube-api-access-lz7mr" (OuterVolumeSpecName: "kube-api-access-lz7mr") pod "734aa3c9-314b-4d19-ae75-6c6469ae244a" (UID: "734aa3c9-314b-4d19-ae75-6c6469ae244a"). InnerVolumeSpecName "kube-api-access-lz7mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:52:14.771579 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.771561    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-kube-api-access-z5gv4" (OuterVolumeSpecName: "kube-api-access-z5gv4") pod "5c82572f-6798-4d50-9dc4-124b7f0bd1d9" (UID: "5c82572f-6798-4d50-9dc4-124b7f0bd1d9"). InnerVolumeSpecName "kube-api-access-z5gv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:52:14.832431 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.832394    2580 generic.go:356] "Generic (PLEG): container finished" podID="734aa3c9-314b-4d19-ae75-6c6469ae244a" containerID="cfd102db5934572cb8c9246d73609fa360b713fa6a991fbf3512f5076efe6114" exitCode=0
Sep 11 13:52:14.832601 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.832442    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"
Sep 11 13:52:14.832601 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.832477    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726" event={"ID":"734aa3c9-314b-4d19-ae75-6c6469ae244a","Type":"ContainerDied","Data":"cfd102db5934572cb8c9246d73609fa360b713fa6a991fbf3512f5076efe6114"}
Sep 11 13:52:14.832601 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.832521    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726" event={"ID":"734aa3c9-314b-4d19-ae75-6c6469ae244a","Type":"ContainerDied","Data":"7460184e3e52d4a50720c65144341e77a5061d21ae756c459bf2f600f4af2f4e"}
Sep 11 13:52:14.832601 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.832539    2580 scope.go:117] "RemoveContainer" containerID="cfd102db5934572cb8c9246d73609fa360b713fa6a991fbf3512f5076efe6114"
Sep 11 13:52:14.834273 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.834248    2580 generic.go:356] "Generic (PLEG): container finished" podID="5c82572f-6798-4d50-9dc4-124b7f0bd1d9" containerID="38449096d2049d480c82fdd268cc053df4f8dc3d6e4a1da7c65729f2dc48ea42" exitCode=0
Sep 11 13:52:14.834381 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.834294    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"
Sep 11 13:52:14.834381 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.834311    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf" event={"ID":"5c82572f-6798-4d50-9dc4-124b7f0bd1d9","Type":"ContainerDied","Data":"38449096d2049d480c82fdd268cc053df4f8dc3d6e4a1da7c65729f2dc48ea42"}
Sep 11 13:52:14.834381 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.834363    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf" event={"ID":"5c82572f-6798-4d50-9dc4-124b7f0bd1d9","Type":"ContainerDied","Data":"a8469f36978aec2a954006ce52bfd253b379d3d4a4484c189700a3b3171cf85d"}
Sep 11 13:52:14.834675 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:52:14.834648    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7616-6dbd49" container="test-curl-7616-6dbd49"
Sep 11 13:52:14.843169 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.843151    2580 scope.go:117] "RemoveContainer" containerID="cfd102db5934572cb8c9246d73609fa360b713fa6a991fbf3512f5076efe6114"
Sep 11 13:52:14.843461 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:52:14.843442    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfd102db5934572cb8c9246d73609fa360b713fa6a991fbf3512f5076efe6114\": container with ID starting with cfd102db5934572cb8c9246d73609fa360b713fa6a991fbf3512f5076efe6114 not found: ID does not exist" containerID="cfd102db5934572cb8c9246d73609fa360b713fa6a991fbf3512f5076efe6114"
Sep 11 13:52:14.843521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.843468    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfd102db5934572cb8c9246d73609fa360b713fa6a991fbf3512f5076efe6114"} err="failed to get container status \"cfd102db5934572cb8c9246d73609fa360b713fa6a991fbf3512f5076efe6114\": rpc error: code = NotFound desc = could not find container \"cfd102db5934572cb8c9246d73609fa360b713fa6a991fbf3512f5076efe6114\": container with ID starting with cfd102db5934572cb8c9246d73609fa360b713fa6a991fbf3512f5076efe6114 not found: ID does not exist"
Sep 11 13:52:14.843521 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.843487    2580 scope.go:117] "RemoveContainer" containerID="38449096d2049d480c82fdd268cc053df4f8dc3d6e4a1da7c65729f2dc48ea42"
Sep 11 13:52:14.851827 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.851812    2580 scope.go:117] "RemoveContainer" containerID="38449096d2049d480c82fdd268cc053df4f8dc3d6e4a1da7c65729f2dc48ea42"
Sep 11 13:52:14.852123 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:52:14.852105    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38449096d2049d480c82fdd268cc053df4f8dc3d6e4a1da7c65729f2dc48ea42\": container with ID starting with 38449096d2049d480c82fdd268cc053df4f8dc3d6e4a1da7c65729f2dc48ea42 not found: ID does not exist" containerID="38449096d2049d480c82fdd268cc053df4f8dc3d6e4a1da7c65729f2dc48ea42"
Sep 11 13:52:14.852192 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.852129    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38449096d2049d480c82fdd268cc053df4f8dc3d6e4a1da7c65729f2dc48ea42"} err="failed to get container status \"38449096d2049d480c82fdd268cc053df4f8dc3d6e4a1da7c65729f2dc48ea42\": rpc error: code = NotFound desc = could not find container \"38449096d2049d480c82fdd268cc053df4f8dc3d6e4a1da7c65729f2dc48ea42\": container with ID starting with 38449096d2049d480c82fdd268cc053df4f8dc3d6e4a1da7c65729f2dc48ea42 not found: ID does not exist"
Sep 11 13:52:14.868527 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.868468    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"]
Sep 11 13:52:14.868876 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.868855    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:14.868993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.868884    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5gv4\" (UniqueName: \"kubernetes.io/projected/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-kube-api-access-z5gv4\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:14.868993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.868914    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz7mr\" (UniqueName: \"kubernetes.io/projected/734aa3c9-314b-4d19-ae75-6c6469ae244a-kube-api-access-lz7mr\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:14.868993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.868942    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/734aa3c9-314b-4d19-ae75-6c6469ae244a-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:14.868993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.868956    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/734aa3c9-314b-4d19-ae75-6c6469ae244a-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:14.868993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.868968    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:14.868993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.868981    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:14.868993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.868993    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c82572f-6798-4d50-9dc4-124b7f0bd1d9-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:14.884568 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.884547    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/distinct-w4-371c38-kserve-587f4ddf8b-cx726"]
Sep 11 13:52:14.909633 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.909606    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"]
Sep 11 13:52:14.920512 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:14.920484    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/simulator-w4-371c38-kserve-68749b6cfd-87jwf"]
Sep 11 13:52:15.316037 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:15.316006    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c82572f-6798-4d50-9dc4-124b7f0bd1d9" path="/var/lib/kubelet/pods/5c82572f-6798-4d50-9dc4-124b7f0bd1d9/volumes"
Sep 11 13:52:15.316429 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:15.316416    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="734aa3c9-314b-4d19-ae75-6c6469ae244a" path="/var/lib/kubelet/pods/734aa3c9-314b-4d19-ae75-6c6469ae244a/volumes"
Sep 11 13:52:15.841489 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:15.841453    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7616-6dbd49" podUID="811711e5-4689-4ee8-97f8-5f4862435e72" containerName="test-curl-7616-6dbd49" containerID="cri-o://a64abab8a1d38e00a2f31039c0e2ed8f28c91555c8d945e5fe178dbfcc008db3" gracePeriod=2
Sep 11 13:52:18.025839 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:18.025810    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-6dbd49"
Sep 11 13:52:18.099445 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:18.099362    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h99sb\" (UniqueName: \"kubernetes.io/projected/811711e5-4689-4ee8-97f8-5f4862435e72-kube-api-access-h99sb\") pod \"811711e5-4689-4ee8-97f8-5f4862435e72\" (UID: \"811711e5-4689-4ee8-97f8-5f4862435e72\") "
Sep 11 13:52:18.101433 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:18.101398    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/811711e5-4689-4ee8-97f8-5f4862435e72-kube-api-access-h99sb" (OuterVolumeSpecName: "kube-api-access-h99sb") pod "811711e5-4689-4ee8-97f8-5f4862435e72" (UID: "811711e5-4689-4ee8-97f8-5f4862435e72"). InnerVolumeSpecName "kube-api-access-h99sb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:52:18.200266 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:18.200228    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h99sb\" (UniqueName: \"kubernetes.io/projected/811711e5-4689-4ee8-97f8-5f4862435e72-kube-api-access-h99sb\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:52:18.861460 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:18.861422    2580 generic.go:356] "Generic (PLEG): container finished" podID="811711e5-4689-4ee8-97f8-5f4862435e72" containerID="a64abab8a1d38e00a2f31039c0e2ed8f28c91555c8d945e5fe178dbfcc008db3" exitCode=137
Sep 11 13:52:18.861627 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:18.861476    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-6dbd49"
Sep 11 13:52:18.861627 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:18.861487    2580 scope.go:117] "RemoveContainer" containerID="a64abab8a1d38e00a2f31039c0e2ed8f28c91555c8d945e5fe178dbfcc008db3"
Sep 11 13:52:18.871505 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:18.871488    2580 scope.go:117] "RemoveContainer" containerID="a64abab8a1d38e00a2f31039c0e2ed8f28c91555c8d945e5fe178dbfcc008db3"
Sep 11 13:52:18.871793 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:52:18.871772    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a64abab8a1d38e00a2f31039c0e2ed8f28c91555c8d945e5fe178dbfcc008db3\": container with ID starting with a64abab8a1d38e00a2f31039c0e2ed8f28c91555c8d945e5fe178dbfcc008db3 not found: ID does not exist" containerID="a64abab8a1d38e00a2f31039c0e2ed8f28c91555c8d945e5fe178dbfcc008db3"
Sep 11 13:52:18.871837 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:18.871802    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a64abab8a1d38e00a2f31039c0e2ed8f28c91555c8d945e5fe178dbfcc008db3"} err="failed to get container status \"a64abab8a1d38e00a2f31039c0e2ed8f28c91555c8d945e5fe178dbfcc008db3\": rpc error: code = NotFound desc = could not find container \"a64abab8a1d38e00a2f31039c0e2ed8f28c91555c8d945e5fe178dbfcc008db3\": container with ID starting with a64abab8a1d38e00a2f31039c0e2ed8f28c91555c8d945e5fe178dbfcc008db3 not found: ID does not exist"
Sep 11 13:52:18.872465 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:52:18.872442    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7616-6dbd49\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="811711e5-4689-4ee8-97f8-5f4862435e72" pod="opendatahub/test-curl-7616-6dbd49"
Sep 11 13:52:19.315445 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:19.315414    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="811711e5-4689-4ee8-97f8-5f4862435e72" path="/var/lib/kubelet/pods/811711e5-4689-4ee8-97f8-5f4862435e72/volumes"
Sep 11 13:52:28.480563 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.480527    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"]
Sep 11 13:52:28.480975 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.480896    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c82572f-6798-4d50-9dc4-124b7f0bd1d9" containerName="main"
Sep 11 13:52:28.480975 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.480926    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c82572f-6798-4d50-9dc4-124b7f0bd1d9" containerName="main"
Sep 11 13:52:28.480975 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.480938    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="734aa3c9-314b-4d19-ae75-6c6469ae244a" containerName="main"
Sep 11 13:52:28.480975 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.480944    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="734aa3c9-314b-4d19-ae75-6c6469ae244a" containerName="main"
Sep 11 13:52:28.480975 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.480950    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="811711e5-4689-4ee8-97f8-5f4862435e72" containerName="test-curl-7616-6dbd49"
Sep 11 13:52:28.480975 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.480956    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="811711e5-4689-4ee8-97f8-5f4862435e72" containerName="test-curl-7616-6dbd49"
Sep 11 13:52:28.481178 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.481014    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="734aa3c9-314b-4d19-ae75-6c6469ae244a" containerName="main"
Sep 11 13:52:28.481178 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.481034    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="811711e5-4689-4ee8-97f8-5f4862435e72" containerName="test-curl-7616-6dbd49"
Sep 11 13:52:28.481178 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.481108    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c82572f-6798-4d50-9dc4-124b7f0bd1d9" containerName="main"
Sep 11 13:52:28.484089 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.484072    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.491342 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.491319    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-0317c3\"/\"openshift-service-ca.crt\""
Sep 11 13:52:28.491693 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.491676    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-0317c3\"/\"sub-test-model-0317c3-kserve-self-signed-certs\""
Sep 11 13:52:28.491799 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.491690    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-0317c3\"/\"default-dockercfg-8bq8v\""
Sep 11 13:52:28.491799 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.491724    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-0317c3\"/\"kube-root-ca.crt\""
Sep 11 13:52:28.507669 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.507642    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"]
Sep 11 13:52:28.593091 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.593052    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-home\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.593235 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.593111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-dshm\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.593235 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.593157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww2tn\" (UniqueName: \"kubernetes.io/projected/cd928603-3c53-4b24-ba79-476b6beb1a63-kube-api-access-ww2tn\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.593235 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.593213    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd928603-3c53-4b24-ba79-476b6beb1a63-tls-certs\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.593360 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.593236    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-model-cache\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.593360 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.593313    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-tmp-dir\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.694674 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.694641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-tmp-dir\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.694866 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.694692    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-home\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.694866 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.694717    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-dshm\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.694866 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.694735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww2tn\" (UniqueName: \"kubernetes.io/projected/cd928603-3c53-4b24-ba79-476b6beb1a63-kube-api-access-ww2tn\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.694866 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.694761    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd928603-3c53-4b24-ba79-476b6beb1a63-tls-certs\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.694866 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.694779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-model-cache\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.695194 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.695137    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-tmp-dir\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.695194 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.695166    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-home\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.695294 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.695218    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-model-cache\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.697056 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.697034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-dshm\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.697345 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.697325    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd928603-3c53-4b24-ba79-476b6beb1a63-tls-certs\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.708253 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.708229    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww2tn\" (UniqueName: \"kubernetes.io/projected/cd928603-3c53-4b24-ba79-476b6beb1a63-kube-api-access-ww2tn\") pod \"sub-test-model-0317c3-kserve-59789cd5b9-xkgwl\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:28.793587 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:28.793501    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:29.076242 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:29.076211    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"]
Sep 11 13:52:29.077139 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:52:29.077111    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd928603_3c53_4b24_ba79_476b6beb1a63.slice/crio-315b27ccd02add4a0e9ee4bc158addca533a940be5c1513fbc5d5dd351e0cd93 WatchSource:0}: Error finding container 315b27ccd02add4a0e9ee4bc158addca533a940be5c1513fbc5d5dd351e0cd93: Status 404 returned error can't find the container with id 315b27ccd02add4a0e9ee4bc158addca533a940be5c1513fbc5d5dd351e0cd93
Sep 11 13:52:29.927070 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:29.927034    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl" event={"ID":"cd928603-3c53-4b24-ba79-476b6beb1a63","Type":"ContainerStarted","Data":"2b404b9bf1f3b64e81f17db89b92911b282b9e089706cd101c1c936fda592d0a"}
Sep 11 13:52:29.927070 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:29.927077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl" event={"ID":"cd928603-3c53-4b24-ba79-476b6beb1a63","Type":"ContainerStarted","Data":"315b27ccd02add4a0e9ee4bc158addca533a940be5c1513fbc5d5dd351e0cd93"}
Sep 11 13:52:29.950604 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:29.950547    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl" podStartSLOduration=1.950531519 podStartE2EDuration="1.950531519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:52:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:52:29.94745809 +0000 UTC m=+1869.165995594" watchObservedRunningTime="2026-09-11 13:52:29.950531519 +0000 UTC m=+1869.169069051"
Sep 11 13:52:33.580794 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:33.580754    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:52:38.794579 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:38.794535    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:38.795058 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:38.794590    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:38.807067 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:38.807042    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:38.990174 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:38.990143    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:52:57.514155 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:52:57.514116    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:53:14.762027 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:14.761990    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:53:25.653355 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:25.653320    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:53:29.107803 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:29.107762    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7616-07244b"]
Sep 11 13:53:29.111103 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:29.111080    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-07244b"
Sep 11 13:53:29.115074 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:29.115044    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7ltrq\""
Sep 11 13:53:29.123079 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:29.123056    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7616-07244b"]
Sep 11 13:53:29.249803 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:29.249761    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf9t4\" (UniqueName: \"kubernetes.io/projected/52a8d9f1-c3ca-400b-a434-a9946def899b-kube-api-access-pf9t4\") pod \"test-curl-7616-07244b\" (UID: \"52a8d9f1-c3ca-400b-a434-a9946def899b\") " pod="opendatahub/test-curl-7616-07244b"
Sep 11 13:53:29.350325 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:29.350294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf9t4\" (UniqueName: \"kubernetes.io/projected/52a8d9f1-c3ca-400b-a434-a9946def899b-kube-api-access-pf9t4\") pod \"test-curl-7616-07244b\" (UID: \"52a8d9f1-c3ca-400b-a434-a9946def899b\") " pod="opendatahub/test-curl-7616-07244b"
Sep 11 13:53:29.362310 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:29.362243    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf9t4\" (UniqueName: \"kubernetes.io/projected/52a8d9f1-c3ca-400b-a434-a9946def899b-kube-api-access-pf9t4\") pod \"test-curl-7616-07244b\" (UID: \"52a8d9f1-c3ca-400b-a434-a9946def899b\") " pod="opendatahub/test-curl-7616-07244b"
Sep 11 13:53:29.420707 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:29.420671    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-07244b"
Sep 11 13:53:29.695323 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:29.695300    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7616-07244b"]
Sep 11 13:53:29.698349 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:53:29.698319    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52a8d9f1_c3ca_400b_a434_a9946def899b.slice/crio-7cd4dd87c3b5f86c3344ce044c1d91ed98d0e31ca0776822c98016b47b20cbba WatchSource:0}: Error finding container 7cd4dd87c3b5f86c3344ce044c1d91ed98d0e31ca0776822c98016b47b20cbba: Status 404 returned error can't find the container with id 7cd4dd87c3b5f86c3344ce044c1d91ed98d0e31ca0776822c98016b47b20cbba
Sep 11 13:53:30.270183 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:30.270145    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7616-07244b" event={"ID":"52a8d9f1-c3ca-400b-a434-a9946def899b","Type":"ContainerStarted","Data":"4e6c0b347b13598c81f5001ebb6f81f5e9d8a10302b3ebb9350f7b2b07322ad8"}
Sep 11 13:53:30.270183 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:30.270189    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7616-07244b" event={"ID":"52a8d9f1-c3ca-400b-a434-a9946def899b","Type":"ContainerStarted","Data":"7cd4dd87c3b5f86c3344ce044c1d91ed98d0e31ca0776822c98016b47b20cbba"}
Sep 11 13:53:30.291248 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:30.291191    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7616-07244b" podStartSLOduration=1.291176261 podStartE2EDuration="1.291176261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:30.288951246 +0000 UTC m=+1929.507488739" watchObservedRunningTime="2026-09-11 13:53:30.291176261 +0000 UTC m=+1929.509713753"
Sep 11 13:53:31.001251 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:31.001207    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7616-07244b"]
Sep 11 13:53:31.010581 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:31.010546    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7616-07244b"]
Sep 11 13:53:31.132807 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:31.132769    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7616-4c50b8"]
Sep 11 13:53:31.133206 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:31.133190    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52a8d9f1-c3ca-400b-a434-a9946def899b" containerName="test-curl-7616-07244b"
Sep 11 13:53:31.133254 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:31.133207    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="52a8d9f1-c3ca-400b-a434-a9946def899b" containerName="test-curl-7616-07244b"
Sep 11 13:53:31.133290 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:31.133265    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="52a8d9f1-c3ca-400b-a434-a9946def899b" containerName="test-curl-7616-07244b"
Sep 11 13:53:31.136323 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:31.136307    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-4c50b8"
Sep 11 13:53:31.145690 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:31.145660    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7616-4c50b8"]
Sep 11 13:53:31.269524 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:31.269464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psg6w\" (UniqueName: \"kubernetes.io/projected/bcd7c9fb-ff44-4789-939b-07a53ff7cd5f-kube-api-access-psg6w\") pod \"test-curl-7616-4c50b8\" (UID: \"bcd7c9fb-ff44-4789-939b-07a53ff7cd5f\") " pod="opendatahub/test-curl-7616-4c50b8"
Sep 11 13:53:31.275311 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:53:31.275283    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7616-07244b" container="test-curl-7616-07244b"
Sep 11 13:53:31.370621 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:31.370585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psg6w\" (UniqueName: \"kubernetes.io/projected/bcd7c9fb-ff44-4789-939b-07a53ff7cd5f-kube-api-access-psg6w\") pod \"test-curl-7616-4c50b8\" (UID: \"bcd7c9fb-ff44-4789-939b-07a53ff7cd5f\") " pod="opendatahub/test-curl-7616-4c50b8"
Sep 11 13:53:31.387772 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:31.387687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psg6w\" (UniqueName: \"kubernetes.io/projected/bcd7c9fb-ff44-4789-939b-07a53ff7cd5f-kube-api-access-psg6w\") pod \"test-curl-7616-4c50b8\" (UID: \"bcd7c9fb-ff44-4789-939b-07a53ff7cd5f\") " pod="opendatahub/test-curl-7616-4c50b8"
Sep 11 13:53:31.446721 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:31.446681    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-4c50b8"
Sep 11 13:53:31.723671 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:31.723644    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7616-4c50b8"]
Sep 11 13:53:31.725559 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:53:31.725529    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbcd7c9fb_ff44_4789_939b_07a53ff7cd5f.slice/crio-6ee0ac69f67df4fe9412d3f5606fee4ac31735c326703fe92917b81ba1958057 WatchSource:0}: Error finding container 6ee0ac69f67df4fe9412d3f5606fee4ac31735c326703fe92917b81ba1958057: Status 404 returned error can't find the container with id 6ee0ac69f67df4fe9412d3f5606fee4ac31735c326703fe92917b81ba1958057
Sep 11 13:53:32.283193 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:32.283158    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7616-4c50b8" event={"ID":"bcd7c9fb-ff44-4789-939b-07a53ff7cd5f","Type":"ContainerStarted","Data":"82b6d12120a50962b17a20a74a5c25f7f78cde4dfa9488036a4fb1d9f97b61c2"}
Sep 11 13:53:32.283605 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:32.283199    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7616-4c50b8" event={"ID":"bcd7c9fb-ff44-4789-939b-07a53ff7cd5f","Type":"ContainerStarted","Data":"6ee0ac69f67df4fe9412d3f5606fee4ac31735c326703fe92917b81ba1958057"}
Sep 11 13:53:32.283605 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:32.283329    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7616-07244b" podUID="52a8d9f1-c3ca-400b-a434-a9946def899b" containerName="test-curl-7616-07244b" containerID="cri-o://4e6c0b347b13598c81f5001ebb6f81f5e9d8a10302b3ebb9350f7b2b07322ad8" gracePeriod=2
Sep 11 13:53:32.303335 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:32.303286    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7616-4c50b8" podStartSLOduration=1.303270989 podStartE2EDuration="1.303270989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:32.302014761 +0000 UTC m=+1931.520552266" watchObservedRunningTime="2026-09-11 13:53:32.303270989 +0000 UTC m=+1931.521808481"
Sep 11 13:53:32.913855 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:32.913806    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7616-4c50b8"]
Sep 11 13:53:32.920883 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:32.920853    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7616-4c50b8"]
Sep 11 13:53:33.289292 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:53:33.289259    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7616-4c50b8" container="test-curl-7616-4c50b8"
Sep 11 13:53:34.295183 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:34.295148    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7616-4c50b8" podUID="bcd7c9fb-ff44-4789-939b-07a53ff7cd5f" containerName="test-curl-7616-4c50b8" containerID="cri-o://82b6d12120a50962b17a20a74a5c25f7f78cde4dfa9488036a4fb1d9f97b61c2" gracePeriod=2
Sep 11 13:53:34.427991 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:34.427966    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-07244b"
Sep 11 13:53:34.601998 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:34.601966    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf9t4\" (UniqueName: \"kubernetes.io/projected/52a8d9f1-c3ca-400b-a434-a9946def899b-kube-api-access-pf9t4\") pod \"52a8d9f1-c3ca-400b-a434-a9946def899b\" (UID: \"52a8d9f1-c3ca-400b-a434-a9946def899b\") "
Sep 11 13:53:34.604026 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:34.603992    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52a8d9f1-c3ca-400b-a434-a9946def899b-kube-api-access-pf9t4" (OuterVolumeSpecName: "kube-api-access-pf9t4") pod "52a8d9f1-c3ca-400b-a434-a9946def899b" (UID: "52a8d9f1-c3ca-400b-a434-a9946def899b"). InnerVolumeSpecName "kube-api-access-pf9t4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:53:34.703261 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:34.703224    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf9t4\" (UniqueName: \"kubernetes.io/projected/52a8d9f1-c3ca-400b-a434-a9946def899b-kube-api-access-pf9t4\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:53:34.984846 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:34.984764    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"]
Sep 11 13:53:35.301590 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:35.301502    2580 generic.go:356] "Generic (PLEG): container finished" podID="52a8d9f1-c3ca-400b-a434-a9946def899b" containerID="4e6c0b347b13598c81f5001ebb6f81f5e9d8a10302b3ebb9350f7b2b07322ad8" exitCode=137
Sep 11 13:53:35.301590 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:35.301543    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-07244b"
Sep 11 13:53:35.301590 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:35.301583    2580 scope.go:117] "RemoveContainer" containerID="4e6c0b347b13598c81f5001ebb6f81f5e9d8a10302b3ebb9350f7b2b07322ad8"
Sep 11 13:53:35.311259 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:35.311235    2580 scope.go:117] "RemoveContainer" containerID="4e6c0b347b13598c81f5001ebb6f81f5e9d8a10302b3ebb9350f7b2b07322ad8"
Sep 11 13:53:35.311538 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:53:35.311517    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e6c0b347b13598c81f5001ebb6f81f5e9d8a10302b3ebb9350f7b2b07322ad8\": container with ID starting with 4e6c0b347b13598c81f5001ebb6f81f5e9d8a10302b3ebb9350f7b2b07322ad8 not found: ID does not exist" containerID="4e6c0b347b13598c81f5001ebb6f81f5e9d8a10302b3ebb9350f7b2b07322ad8"
Sep 11 13:53:35.311619 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:35.311547    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e6c0b347b13598c81f5001ebb6f81f5e9d8a10302b3ebb9350f7b2b07322ad8"} err="failed to get container status \"4e6c0b347b13598c81f5001ebb6f81f5e9d8a10302b3ebb9350f7b2b07322ad8\": rpc error: code = NotFound desc = could not find container \"4e6c0b347b13598c81f5001ebb6f81f5e9d8a10302b3ebb9350f7b2b07322ad8\": container with ID starting with 4e6c0b347b13598c81f5001ebb6f81f5e9d8a10302b3ebb9350f7b2b07322ad8 not found: ID does not exist"
Sep 11 13:53:35.312810 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:53:35.312786    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7616-07244b\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="52a8d9f1-c3ca-400b-a434-a9946def899b" pod="opendatahub/test-curl-7616-07244b"
Sep 11 13:53:35.315445 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:35.315425    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52a8d9f1-c3ca-400b-a434-a9946def899b" path="/var/lib/kubelet/pods/52a8d9f1-c3ca-400b-a434-a9946def899b/volumes"
Sep 11 13:53:36.488969 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:36.488945    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-4c50b8"
Sep 11 13:53:36.517242 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:36.517208    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psg6w\" (UniqueName: \"kubernetes.io/projected/bcd7c9fb-ff44-4789-939b-07a53ff7cd5f-kube-api-access-psg6w\") pod \"bcd7c9fb-ff44-4789-939b-07a53ff7cd5f\" (UID: \"bcd7c9fb-ff44-4789-939b-07a53ff7cd5f\") "
Sep 11 13:53:36.519169 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:36.519148    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcd7c9fb-ff44-4789-939b-07a53ff7cd5f-kube-api-access-psg6w" (OuterVolumeSpecName: "kube-api-access-psg6w") pod "bcd7c9fb-ff44-4789-939b-07a53ff7cd5f" (UID: "bcd7c9fb-ff44-4789-939b-07a53ff7cd5f"). InnerVolumeSpecName "kube-api-access-psg6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:53:36.618398 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:36.618361    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psg6w\" (UniqueName: \"kubernetes.io/projected/bcd7c9fb-ff44-4789-939b-07a53ff7cd5f-kube-api-access-psg6w\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:53:37.316398 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:37.316357    2580 generic.go:356] "Generic (PLEG): container finished" podID="bcd7c9fb-ff44-4789-939b-07a53ff7cd5f" containerID="82b6d12120a50962b17a20a74a5c25f7f78cde4dfa9488036a4fb1d9f97b61c2" exitCode=137
Sep 11 13:53:37.316571 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:37.316466    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7616-4c50b8"
Sep 11 13:53:37.316571 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:37.316514    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcd7c9fb-ff44-4789-939b-07a53ff7cd5f" path="/var/lib/kubelet/pods/bcd7c9fb-ff44-4789-939b-07a53ff7cd5f/volumes"
Sep 11 13:53:37.316892 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:37.316879    2580 scope.go:117] "RemoveContainer" containerID="82b6d12120a50962b17a20a74a5c25f7f78cde4dfa9488036a4fb1d9f97b61c2"
Sep 11 13:53:37.326598 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:37.326581    2580 scope.go:117] "RemoveContainer" containerID="82b6d12120a50962b17a20a74a5c25f7f78cde4dfa9488036a4fb1d9f97b61c2"
Sep 11 13:53:37.326914 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:53:37.326880    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82b6d12120a50962b17a20a74a5c25f7f78cde4dfa9488036a4fb1d9f97b61c2\": container with ID starting with 82b6d12120a50962b17a20a74a5c25f7f78cde4dfa9488036a4fb1d9f97b61c2 not found: ID does not exist" containerID="82b6d12120a50962b17a20a74a5c25f7f78cde4dfa9488036a4fb1d9f97b61c2"
Sep 11 13:53:37.326982 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:37.326938    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82b6d12120a50962b17a20a74a5c25f7f78cde4dfa9488036a4fb1d9f97b61c2"} err="failed to get container status \"82b6d12120a50962b17a20a74a5c25f7f78cde4dfa9488036a4fb1d9f97b61c2\": rpc error: code = NotFound desc = could not find container \"82b6d12120a50962b17a20a74a5c25f7f78cde4dfa9488036a4fb1d9f97b61c2\": container with ID starting with 82b6d12120a50962b17a20a74a5c25f7f78cde4dfa9488036a4fb1d9f97b61c2 not found: ID does not exist"
Sep 11 13:53:38.173093 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:38.173061    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:53:50.004070 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.004007    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl" podUID="cd928603-3c53-4b24-ba79-476b6beb1a63" containerName="main" containerID="cri-o://2b404b9bf1f3b64e81f17db89b92911b282b9e089706cd101c1c936fda592d0a" gracePeriod=45
Sep 11 13:53:50.183373 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.183340    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:53:50.266091 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.266034    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:53:50.342562 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.342524    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-home\") pod \"cd928603-3c53-4b24-ba79-476b6beb1a63\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") "
Sep 11 13:53:50.342759 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.342577    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd928603-3c53-4b24-ba79-476b6beb1a63-tls-certs\") pod \"cd928603-3c53-4b24-ba79-476b6beb1a63\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") "
Sep 11 13:53:50.342759 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.342607    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww2tn\" (UniqueName: \"kubernetes.io/projected/cd928603-3c53-4b24-ba79-476b6beb1a63-kube-api-access-ww2tn\") pod \"cd928603-3c53-4b24-ba79-476b6beb1a63\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") "
Sep 11 13:53:50.342759 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.342664    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-dshm\") pod \"cd928603-3c53-4b24-ba79-476b6beb1a63\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") "
Sep 11 13:53:50.342759 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.342707    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-model-cache\") pod \"cd928603-3c53-4b24-ba79-476b6beb1a63\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") "
Sep 11 13:53:50.342759 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.342739    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-tmp-dir\") pod \"cd928603-3c53-4b24-ba79-476b6beb1a63\" (UID: \"cd928603-3c53-4b24-ba79-476b6beb1a63\") "
Sep 11 13:53:50.343189 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.342849    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-home" (OuterVolumeSpecName: "home") pod "cd928603-3c53-4b24-ba79-476b6beb1a63" (UID: "cd928603-3c53-4b24-ba79-476b6beb1a63"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:53:50.343189 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.343066    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-model-cache" (OuterVolumeSpecName: "model-cache") pod "cd928603-3c53-4b24-ba79-476b6beb1a63" (UID: "cd928603-3c53-4b24-ba79-476b6beb1a63"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:53:50.343189 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.343082    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cd928603-3c53-4b24-ba79-476b6beb1a63" (UID: "cd928603-3c53-4b24-ba79-476b6beb1a63"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:53:50.343189 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.343097    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:53:50.345079 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.345056    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd928603-3c53-4b24-ba79-476b6beb1a63-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cd928603-3c53-4b24-ba79-476b6beb1a63" (UID: "cd928603-3c53-4b24-ba79-476b6beb1a63"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:53:50.345079 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.345063    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-dshm" (OuterVolumeSpecName: "dshm") pod "cd928603-3c53-4b24-ba79-476b6beb1a63" (UID: "cd928603-3c53-4b24-ba79-476b6beb1a63"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:53:50.345249 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.345202    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd928603-3c53-4b24-ba79-476b6beb1a63-kube-api-access-ww2tn" (OuterVolumeSpecName: "kube-api-access-ww2tn") pod "cd928603-3c53-4b24-ba79-476b6beb1a63" (UID: "cd928603-3c53-4b24-ba79-476b6beb1a63"). InnerVolumeSpecName "kube-api-access-ww2tn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:53:50.392467 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.392428    2580 generic.go:356] "Generic (PLEG): container finished" podID="cd928603-3c53-4b24-ba79-476b6beb1a63" containerID="2b404b9bf1f3b64e81f17db89b92911b282b9e089706cd101c1c936fda592d0a" exitCode=0
Sep 11 13:53:50.392659 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.392475    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"
Sep 11 13:53:50.392659 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.392508    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl" event={"ID":"cd928603-3c53-4b24-ba79-476b6beb1a63","Type":"ContainerDied","Data":"2b404b9bf1f3b64e81f17db89b92911b282b9e089706cd101c1c936fda592d0a"}
Sep 11 13:53:50.392659 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.392549    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl" event={"ID":"cd928603-3c53-4b24-ba79-476b6beb1a63","Type":"ContainerDied","Data":"315b27ccd02add4a0e9ee4bc158addca533a940be5c1513fbc5d5dd351e0cd93"}
Sep 11 13:53:50.392659 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.392566    2580 scope.go:117] "RemoveContainer" containerID="2b404b9bf1f3b64e81f17db89b92911b282b9e089706cd101c1c936fda592d0a"
Sep 11 13:53:50.402656 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.402639    2580 scope.go:117] "RemoveContainer" containerID="2b404b9bf1f3b64e81f17db89b92911b282b9e089706cd101c1c936fda592d0a"
Sep 11 13:53:50.402894 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:53:50.402868    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b404b9bf1f3b64e81f17db89b92911b282b9e089706cd101c1c936fda592d0a\": container with ID starting with 2b404b9bf1f3b64e81f17db89b92911b282b9e089706cd101c1c936fda592d0a not found: ID does not exist" containerID="2b404b9bf1f3b64e81f17db89b92911b282b9e089706cd101c1c936fda592d0a"
Sep 11 13:53:50.402949 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.402895    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b404b9bf1f3b64e81f17db89b92911b282b9e089706cd101c1c936fda592d0a"} err="failed to get container status \"2b404b9bf1f3b64e81f17db89b92911b282b9e089706cd101c1c936fda592d0a\": rpc error: code = NotFound desc = could not find container \"2b404b9bf1f3b64e81f17db89b92911b282b9e089706cd101c1c936fda592d0a\": container with ID starting with 2b404b9bf1f3b64e81f17db89b92911b282b9e089706cd101c1c936fda592d0a not found: ID does not exist"
Sep 11 13:53:50.429198 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.429170    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"]
Sep 11 13:53:50.435854 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.435834    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/sub-test-model-0317c3-kserve-59789cd5b9-xkgwl"]
Sep 11 13:53:50.444293 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.444278    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:53:50.444354 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.444295    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:53:50.444354 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.444304    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd928603-3c53-4b24-ba79-476b6beb1a63-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:53:50.444354 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.444311    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd928603-3c53-4b24-ba79-476b6beb1a63-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:53:50.444354 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:50.444319    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww2tn\" (UniqueName: \"kubernetes.io/projected/cd928603-3c53-4b24-ba79-476b6beb1a63-kube-api-access-ww2tn\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:53:51.316875 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:51.316833    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd928603-3c53-4b24-ba79-476b6beb1a63" path="/var/lib/kubelet/pods/cd928603-3c53-4b24-ba79-476b6beb1a63/volumes"
Sep 11 13:53:52.664043 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.664008    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"]
Sep 11 13:53:52.664522 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.664506    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcd7c9fb-ff44-4789-939b-07a53ff7cd5f" containerName="test-curl-7616-4c50b8"
Sep 11 13:53:52.664579 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.664524    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcd7c9fb-ff44-4789-939b-07a53ff7cd5f" containerName="test-curl-7616-4c50b8"
Sep 11 13:53:52.664579 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.664552    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd928603-3c53-4b24-ba79-476b6beb1a63" containerName="main"
Sep 11 13:53:52.664579 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.664558    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd928603-3c53-4b24-ba79-476b6beb1a63" containerName="main"
Sep 11 13:53:52.664676 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.664635    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcd7c9fb-ff44-4789-939b-07a53ff7cd5f" containerName="test-curl-7616-4c50b8"
Sep 11 13:53:52.664676 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.664643    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd928603-3c53-4b24-ba79-476b6beb1a63" containerName="main"
Sep 11 13:53:52.669319 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.669298    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.674869 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.674848    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-0317c3\"/\"default-dockercfg-8bq8v\""
Sep 11 13:53:52.675887 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.675824    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-0317c3\"/\"openshift-service-ca.crt\""
Sep 11 13:53:52.676143 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.676122    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-0317c3\"/\"kube-root-ca.crt\""
Sep 11 13:53:52.676239 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.676154    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-0317c3\"/\"rate-test-model-0317c3-kserve-self-signed-certs\""
Sep 11 13:53:52.681313 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.681289    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"]
Sep 11 13:53:52.765114 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.765074    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsjd9\" (UniqueName: \"kubernetes.io/projected/db40d691-ba5a-4787-bf1f-7e94249eede4-kube-api-access-gsjd9\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.765114 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.765120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-dshm\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.765394 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.765150    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-tmp-dir\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.765394 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.765246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db40d691-ba5a-4787-bf1f-7e94249eede4-tls-certs\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.765394 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.765270    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-model-cache\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.765394 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.765309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-home\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.865779 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.865744    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db40d691-ba5a-4787-bf1f-7e94249eede4-tls-certs\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.865779 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.865783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-model-cache\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.866027 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.865836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-home\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.866027 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.865865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsjd9\" (UniqueName: \"kubernetes.io/projected/db40d691-ba5a-4787-bf1f-7e94249eede4-kube-api-access-gsjd9\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.866027 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.865886    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-dshm\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.866027 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.865939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-tmp-dir\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.866298 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.866278    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-model-cache\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.866371 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.866293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-home\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.866413 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.866380    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-tmp-dir\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.868169 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.868148    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-dshm\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.868435 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.868417    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db40d691-ba5a-4787-bf1f-7e94249eede4-tls-certs\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.878741 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.878715    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsjd9\" (UniqueName: \"kubernetes.io/projected/db40d691-ba5a-4787-bf1f-7e94249eede4-kube-api-access-gsjd9\") pod \"rate-test-model-0317c3-kserve-6b9d746755-mq4bh\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:52.978861 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:52.978769    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:53:53.255854 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:53.255826    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"]
Sep 11 13:53:53.256735 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:53:53.256705    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb40d691_ba5a_4787_bf1f_7e94249eede4.slice/crio-c6fd9af2009eb6341f669be63dfde2ae4119fb98e0569d8e0f765b43b66d99d4 WatchSource:0}: Error finding container c6fd9af2009eb6341f669be63dfde2ae4119fb98e0569d8e0f765b43b66d99d4: Status 404 returned error can't find the container with id c6fd9af2009eb6341f669be63dfde2ae4119fb98e0569d8e0f765b43b66d99d4
Sep 11 13:53:53.411536 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:53.411492    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh" event={"ID":"db40d691-ba5a-4787-bf1f-7e94249eede4","Type":"ContainerStarted","Data":"e32a9e266b39e8a6654929edf6e4009d0b374c3447ad731bd3785628eaa06344"}
Sep 11 13:53:53.411701 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:53.411545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh" event={"ID":"db40d691-ba5a-4787-bf1f-7e94249eede4","Type":"ContainerStarted","Data":"c6fd9af2009eb6341f669be63dfde2ae4119fb98e0569d8e0f765b43b66d99d4"}
Sep 11 13:53:53.435163 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:53:53.435111    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh" podStartSLOduration=1.4350963540000001 podStartE2EDuration="1.435096354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:53:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:53:53.432835141 +0000 UTC m=+1952.651372632" watchObservedRunningTime="2026-09-11 13:53:53.435096354 +0000 UTC m=+1952.653633849"
Sep 11 13:54:02.859076 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:54:02.859038    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:54:02.979755 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:54:02.979717    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:54:02.979949 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:54:02.979767    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:54:02.992025 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:54:02.991998    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:54:03.481602 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:54:03.481570    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:54:34.652622 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:54:34.652584    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:54:55.272299 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:54:55.272259    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:55:06.117241 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:06.117201    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:55:11.269315 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:11.269275    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"]
Sep 11 13:55:20.434984 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:20.434943    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"]
Sep 11 13:55:20.438582 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:20.438563    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"
Sep 11 13:55:20.441081 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:20.441059    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-264b08-dockercfg-zj99k\""
Sep 11 13:55:20.441452 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:20.441434    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-264b08\""
Sep 11 13:55:20.453883 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:20.453847    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"]
Sep 11 13:55:20.467241 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:20.467203    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07e30bc5-9bc0-47ab-b4e3-23dc72d19285-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc\" (UID: \"07e30bc5-9bc0-47ab-b4e3-23dc72d19285\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"
Sep 11 13:55:20.467378 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:20.467274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6wqh\" (UniqueName: \"kubernetes.io/projected/07e30bc5-9bc0-47ab-b4e3-23dc72d19285-kube-api-access-v6wqh\") pod \"payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc\" (UID: \"07e30bc5-9bc0-47ab-b4e3-23dc72d19285\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"
Sep 11 13:55:20.568513 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:20.568478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07e30bc5-9bc0-47ab-b4e3-23dc72d19285-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc\" (UID: \"07e30bc5-9bc0-47ab-b4e3-23dc72d19285\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"
Sep 11 13:55:20.568696 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:20.568537    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6wqh\" (UniqueName: \"kubernetes.io/projected/07e30bc5-9bc0-47ab-b4e3-23dc72d19285-kube-api-access-v6wqh\") pod \"payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc\" (UID: \"07e30bc5-9bc0-47ab-b4e3-23dc72d19285\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"
Sep 11 13:55:20.569300 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:20.569280    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07e30bc5-9bc0-47ab-b4e3-23dc72d19285-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc\" (UID: \"07e30bc5-9bc0-47ab-b4e3-23dc72d19285\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"
Sep 11 13:55:20.578523 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:20.578500    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6wqh\" (UniqueName: \"kubernetes.io/projected/07e30bc5-9bc0-47ab-b4e3-23dc72d19285-kube-api-access-v6wqh\") pod \"payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc\" (UID: \"07e30bc5-9bc0-47ab-b4e3-23dc72d19285\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"
Sep 11 13:55:20.750586 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:20.750500    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"
Sep 11 13:55:21.025370 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:21.025340    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"]
Sep 11 13:55:21.028201 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:55:21.028177    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07e30bc5_9bc0_47ab_b4e3_23dc72d19285.slice/crio-bf513ceb9dcb4c38f5610bc00cb5c9ab2a2c464a581d26def660fe6f0d576843 WatchSource:0}: Error finding container bf513ceb9dcb4c38f5610bc00cb5c9ab2a2c464a581d26def660fe6f0d576843: Status 404 returned error can't find the container with id bf513ceb9dcb4c38f5610bc00cb5c9ab2a2c464a581d26def660fe6f0d576843
Sep 11 13:55:21.912916 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:21.912869    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc" event={"ID":"07e30bc5-9bc0-47ab-b4e3-23dc72d19285","Type":"ContainerStarted","Data":"a749b9922daa50ab61a88d6b971ed3a708fa9098e2cff5238de94461f123a86d"}
Sep 11 13:55:21.913295 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:21.912951    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"
Sep 11 13:55:21.913295 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:21.912977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc" event={"ID":"07e30bc5-9bc0-47ab-b4e3-23dc72d19285","Type":"ContainerStarted","Data":"bf513ceb9dcb4c38f5610bc00cb5c9ab2a2c464a581d26def660fe6f0d576843"}
Sep 11 13:55:21.935307 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:21.935255    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc" podStartSLOduration=1.9352391359999999 podStartE2EDuration="1.935239136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:55:21.933706278 +0000 UTC m=+2041.152243782" watchObservedRunningTime="2026-09-11 13:55:21.935239136 +0000 UTC m=+2041.153776630"
Sep 11 13:55:24.279680 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:24.279642    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:55:26.288353 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.288298    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh" podUID="db40d691-ba5a-4787-bf1f-7e94249eede4" containerName="main" containerID="cri-o://e32a9e266b39e8a6654929edf6e4009d0b374c3447ad731bd3785628eaa06344" gracePeriod=45
Sep 11 13:55:26.533445 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.533417    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:55:26.622010 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.621978    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsjd9\" (UniqueName: \"kubernetes.io/projected/db40d691-ba5a-4787-bf1f-7e94249eede4-kube-api-access-gsjd9\") pod \"db40d691-ba5a-4787-bf1f-7e94249eede4\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") "
Sep 11 13:55:26.622010 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.622016    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-tmp-dir\") pod \"db40d691-ba5a-4787-bf1f-7e94249eede4\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") "
Sep 11 13:55:26.622230 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.622056    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-model-cache\") pod \"db40d691-ba5a-4787-bf1f-7e94249eede4\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") "
Sep 11 13:55:26.622230 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.622077    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-home\") pod \"db40d691-ba5a-4787-bf1f-7e94249eede4\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") "
Sep 11 13:55:26.622230 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.622121    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db40d691-ba5a-4787-bf1f-7e94249eede4-tls-certs\") pod \"db40d691-ba5a-4787-bf1f-7e94249eede4\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") "
Sep 11 13:55:26.622230 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.622166    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-dshm\") pod \"db40d691-ba5a-4787-bf1f-7e94249eede4\" (UID: \"db40d691-ba5a-4787-bf1f-7e94249eede4\") "
Sep 11 13:55:26.622440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.622314    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "db40d691-ba5a-4787-bf1f-7e94249eede4" (UID: "db40d691-ba5a-4787-bf1f-7e94249eede4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:55:26.622440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.622321    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-model-cache" (OuterVolumeSpecName: "model-cache") pod "db40d691-ba5a-4787-bf1f-7e94249eede4" (UID: "db40d691-ba5a-4787-bf1f-7e94249eede4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:55:26.622440 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.622384    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-home" (OuterVolumeSpecName: "home") pod "db40d691-ba5a-4787-bf1f-7e94249eede4" (UID: "db40d691-ba5a-4787-bf1f-7e94249eede4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:55:26.622603 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.622462    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:55:26.622603 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.622476    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:55:26.622603 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.622484    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:55:26.624228 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.624200    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-dshm" (OuterVolumeSpecName: "dshm") pod "db40d691-ba5a-4787-bf1f-7e94249eede4" (UID: "db40d691-ba5a-4787-bf1f-7e94249eede4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:55:26.624228 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.624206    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db40d691-ba5a-4787-bf1f-7e94249eede4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "db40d691-ba5a-4787-bf1f-7e94249eede4" (UID: "db40d691-ba5a-4787-bf1f-7e94249eede4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:55:26.624404 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.624294    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db40d691-ba5a-4787-bf1f-7e94249eede4-kube-api-access-gsjd9" (OuterVolumeSpecName: "kube-api-access-gsjd9") pod "db40d691-ba5a-4787-bf1f-7e94249eede4" (UID: "db40d691-ba5a-4787-bf1f-7e94249eede4"). InnerVolumeSpecName "kube-api-access-gsjd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:55:26.723753 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.723720    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db40d691-ba5a-4787-bf1f-7e94249eede4-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:55:26.723753 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.723746    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db40d691-ba5a-4787-bf1f-7e94249eede4-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:55:26.723753 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.723754    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsjd9\" (UniqueName: \"kubernetes.io/projected/db40d691-ba5a-4787-bf1f-7e94249eede4-kube-api-access-gsjd9\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:55:26.944107 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.944018    2580 generic.go:356] "Generic (PLEG): container finished" podID="db40d691-ba5a-4787-bf1f-7e94249eede4" containerID="e32a9e266b39e8a6654929edf6e4009d0b374c3447ad731bd3785628eaa06344" exitCode=0
Sep 11 13:55:26.944107 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.944063    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"
Sep 11 13:55:26.944316 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.944102    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh" event={"ID":"db40d691-ba5a-4787-bf1f-7e94249eede4","Type":"ContainerDied","Data":"e32a9e266b39e8a6654929edf6e4009d0b374c3447ad731bd3785628eaa06344"}
Sep 11 13:55:26.944316 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.944147    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh" event={"ID":"db40d691-ba5a-4787-bf1f-7e94249eede4","Type":"ContainerDied","Data":"c6fd9af2009eb6341f669be63dfde2ae4119fb98e0569d8e0f765b43b66d99d4"}
Sep 11 13:55:26.944316 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.944165    2580 scope.go:117] "RemoveContainer" containerID="e32a9e266b39e8a6654929edf6e4009d0b374c3447ad731bd3785628eaa06344"
Sep 11 13:55:26.953639 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.953621    2580 scope.go:117] "RemoveContainer" containerID="e32a9e266b39e8a6654929edf6e4009d0b374c3447ad731bd3785628eaa06344"
Sep 11 13:55:26.953883 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:55:26.953861    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e32a9e266b39e8a6654929edf6e4009d0b374c3447ad731bd3785628eaa06344\": container with ID starting with e32a9e266b39e8a6654929edf6e4009d0b374c3447ad731bd3785628eaa06344 not found: ID does not exist" containerID="e32a9e266b39e8a6654929edf6e4009d0b374c3447ad731bd3785628eaa06344"
Sep 11 13:55:26.953995 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.953889    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32a9e266b39e8a6654929edf6e4009d0b374c3447ad731bd3785628eaa06344"} err="failed to get container status \"e32a9e266b39e8a6654929edf6e4009d0b374c3447ad731bd3785628eaa06344\": rpc error: code = NotFound desc = could not find container \"e32a9e266b39e8a6654929edf6e4009d0b374c3447ad731bd3785628eaa06344\": container with ID starting with e32a9e266b39e8a6654929edf6e4009d0b374c3447ad731bd3785628eaa06344 not found: ID does not exist"
Sep 11 13:55:26.966155 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.966130    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"]
Sep 11 13:55:26.972478 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:26.972457    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/rate-test-model-0317c3-kserve-6b9d746755-mq4bh"]
Sep 11 13:55:27.315338 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:27.315303    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db40d691-ba5a-4787-bf1f-7e94249eede4" path="/var/lib/kubelet/pods/db40d691-ba5a-4787-bf1f-7e94249eede4/volumes"
Sep 11 13:55:32.920078 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:32.920045    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"
Sep 11 13:55:50.090676 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.090639    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"]
Sep 11 13:55:50.091101 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.091032    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db40d691-ba5a-4787-bf1f-7e94249eede4" containerName="main"
Sep 11 13:55:50.091101 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.091044    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="db40d691-ba5a-4787-bf1f-7e94249eede4" containerName="main"
Sep 11 13:55:50.091184 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.091116    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="db40d691-ba5a-4787-bf1f-7e94249eede4" containerName="main"
Sep 11 13:55:50.095813 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.095793    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"
Sep 11 13:55:50.098286 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.098265    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-a403bc\""
Sep 11 13:55:50.103515 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.103491    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"]
Sep 11 13:55:50.107315 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.107294    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"]
Sep 11 13:55:50.110608 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.110593    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"
Sep 11 13:55:50.113449 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.113428    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-a403bc-dockercfg-msgcc\""
Sep 11 13:55:50.123746 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.123722    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"]
Sep 11 13:55:50.237363 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.237329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc56p\" (UniqueName: \"kubernetes.io/projected/55b268b7-44da-4f02-823e-b60eb2eb3c03-kube-api-access-qc56p\") pod \"payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt\" (UID: \"55b268b7-44da-4f02-823e-b60eb2eb3c03\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"
Sep 11 13:55:50.237529 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.237409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9326e58a-a408-4a5d-8e3a-6116eb3a2381-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7\" (UID: \"9326e58a-a408-4a5d-8e3a-6116eb3a2381\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"
Sep 11 13:55:50.237529 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.237432    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55b268b7-44da-4f02-823e-b60eb2eb3c03-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt\" (UID: \"55b268b7-44da-4f02-823e-b60eb2eb3c03\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"
Sep 11 13:55:50.237529 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.237450    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crtgs\" (UniqueName: \"kubernetes.io/projected/9326e58a-a408-4a5d-8e3a-6116eb3a2381-kube-api-access-crtgs\") pod \"payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7\" (UID: \"9326e58a-a408-4a5d-8e3a-6116eb3a2381\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"
Sep 11 13:55:50.338230 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.338191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55b268b7-44da-4f02-823e-b60eb2eb3c03-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt\" (UID: \"55b268b7-44da-4f02-823e-b60eb2eb3c03\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"
Sep 11 13:55:50.338230 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.338236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crtgs\" (UniqueName: \"kubernetes.io/projected/9326e58a-a408-4a5d-8e3a-6116eb3a2381-kube-api-access-crtgs\") pod \"payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7\" (UID: \"9326e58a-a408-4a5d-8e3a-6116eb3a2381\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"
Sep 11 13:55:50.338441 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.338295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc56p\" (UniqueName: \"kubernetes.io/projected/55b268b7-44da-4f02-823e-b60eb2eb3c03-kube-api-access-qc56p\") pod \"payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt\" (UID: \"55b268b7-44da-4f02-823e-b60eb2eb3c03\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"
Sep 11 13:55:50.338441 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.338367    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9326e58a-a408-4a5d-8e3a-6116eb3a2381-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7\" (UID: \"9326e58a-a408-4a5d-8e3a-6116eb3a2381\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"
Sep 11 13:55:50.338969 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.338950    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55b268b7-44da-4f02-823e-b60eb2eb3c03-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt\" (UID: \"55b268b7-44da-4f02-823e-b60eb2eb3c03\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"
Sep 11 13:55:50.339137 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.339120    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9326e58a-a408-4a5d-8e3a-6116eb3a2381-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7\" (UID: \"9326e58a-a408-4a5d-8e3a-6116eb3a2381\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"
Sep 11 13:55:50.347669 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.347603    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crtgs\" (UniqueName: \"kubernetes.io/projected/9326e58a-a408-4a5d-8e3a-6116eb3a2381-kube-api-access-crtgs\") pod \"payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7\" (UID: \"9326e58a-a408-4a5d-8e3a-6116eb3a2381\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"
Sep 11 13:55:50.348221 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.348197    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc56p\" (UniqueName: \"kubernetes.io/projected/55b268b7-44da-4f02-823e-b60eb2eb3c03-kube-api-access-qc56p\") pod \"payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt\" (UID: \"55b268b7-44da-4f02-823e-b60eb2eb3c03\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"
Sep 11 13:55:50.407704 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.407669    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"
Sep 11 13:55:50.421098 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.421071    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"
Sep 11 13:55:50.683623 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.683591    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"]
Sep 11 13:55:50.684392 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:55:50.684363    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55b268b7_44da_4f02_823e_b60eb2eb3c03.slice/crio-21972e0ec77634e627d1048f1996c2cc3725b46404b25e6075b940d0a8134a2b WatchSource:0}: Error finding container 21972e0ec77634e627d1048f1996c2cc3725b46404b25e6075b940d0a8134a2b: Status 404 returned error can't find the container with id 21972e0ec77634e627d1048f1996c2cc3725b46404b25e6075b940d0a8134a2b
Sep 11 13:55:50.707624 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:50.707601    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"]
Sep 11 13:55:50.709114 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:55:50.709025    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9326e58a_a408_4a5d_8e3a_6116eb3a2381.slice/crio-6a0e80f01f903bbc657a320b93a2cd319515d1ed4ceabec1b59c2dceeb3014cf WatchSource:0}: Error finding container 6a0e80f01f903bbc657a320b93a2cd319515d1ed4ceabec1b59c2dceeb3014cf: Status 404 returned error can't find the container with id 6a0e80f01f903bbc657a320b93a2cd319515d1ed4ceabec1b59c2dceeb3014cf
Sep 11 13:55:51.078981 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:51.078939    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7" event={"ID":"9326e58a-a408-4a5d-8e3a-6116eb3a2381","Type":"ContainerStarted","Data":"f27805378ed7b11c794743e3ce8bd474bcedc1c852a0e6d28de18ddca3fc64d6"}
Sep 11 13:55:51.079178 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:51.079008    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7" event={"ID":"9326e58a-a408-4a5d-8e3a-6116eb3a2381","Type":"ContainerStarted","Data":"6a0e80f01f903bbc657a320b93a2cd319515d1ed4ceabec1b59c2dceeb3014cf"}
Sep 11 13:55:51.079178 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:51.079069    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"
Sep 11 13:55:51.080741 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:51.080713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt" event={"ID":"55b268b7-44da-4f02-823e-b60eb2eb3c03","Type":"ContainerStarted","Data":"412d5f809d6b1656e04812f3af79bc4d77835f04b4378acf3ae668f353d6a1d9"}
Sep 11 13:55:51.080884 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:51.080759    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt" event={"ID":"55b268b7-44da-4f02-823e-b60eb2eb3c03","Type":"ContainerStarted","Data":"21972e0ec77634e627d1048f1996c2cc3725b46404b25e6075b940d0a8134a2b"}
Sep 11 13:55:51.080884 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:51.080786    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"
Sep 11 13:55:51.099421 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:51.099367    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7" podStartSLOduration=1.099351151 podStartE2EDuration="1.099351151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:55:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:55:51.095303681 +0000 UTC m=+2070.313841171" watchObservedRunningTime="2026-09-11 13:55:51.099351151 +0000 UTC m=+2070.317888645"
Sep 11 13:55:51.112188 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:51.112137    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt" podStartSLOduration=1.112122656 podStartE2EDuration="1.112122656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:55:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:55:51.111462664 +0000 UTC m=+2070.330000156" watchObservedRunningTime="2026-09-11 13:55:51.112122656 +0000 UTC m=+2070.330660192"
Sep 11 13:55:54.159171 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:55:54.159135    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:56:02.088701 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:02.088668    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"
Sep 11 13:56:02.089340 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:02.088737    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"
Sep 11 13:56:21.326892 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:21.326863    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7_9326e58a-a408-4a5d-8e3a-6116eb3a2381/payload-processing/0.log"
Sep 11 13:56:21.476140 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:21.476109    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:56:21.484295 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:21.484272    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 13:56:21.783232 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:21.783152    2580 scope.go:117] "RemoveContainer" containerID="658b66883cd336f596831017e96de0b920767d3ec49fec643bdcbfb63295a68a"
Sep 11 13:56:21.792461 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:21.792444    2580 scope.go:117] "RemoveContainer" containerID="3cdc1510e65ddaea0ecf318915ab7e39e3e52581bf8218660766e737b4842a91"
Sep 11 13:56:25.538458 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.538426    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"]
Sep 11 13:56:25.543229 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.543207    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.547750 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.547722    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-264b08\"/\"ipp-route-264b08-kserve-self-signed-certs\""
Sep 11 13:56:25.547897 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.547729    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-264b08\"/\"kube-root-ca.crt\""
Sep 11 13:56:25.548551 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.548530    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-264b08\"/\"default-dockercfg-xn2g5\""
Sep 11 13:56:25.548654 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.548555    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-264b08\"/\"openshift-service-ca.crt\""
Sep 11 13:56:25.551622 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.551601    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"]
Sep 11 13:56:25.661048 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.661016    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-tmp-dir\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.661226 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.661071    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-dshm\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.661226 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.661143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-home\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.661226 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.661196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f07a290-4c91-464b-a84a-1c1e154da3c0-tls-certs\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.661336 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.661245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpnm4\" (UniqueName: \"kubernetes.io/projected/8f07a290-4c91-464b-a84a-1c1e154da3c0-kube-api-access-bpnm4\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.661336 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.661276    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-model-cache\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.762092 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.762055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f07a290-4c91-464b-a84a-1c1e154da3c0-tls-certs\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.762300 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.762112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpnm4\" (UniqueName: \"kubernetes.io/projected/8f07a290-4c91-464b-a84a-1c1e154da3c0-kube-api-access-bpnm4\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.762300 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.762146    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-model-cache\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.762300 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.762199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-tmp-dir\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.762300 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.762250    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-dshm\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.762300 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.762287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-home\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.762633 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.762598    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-model-cache\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.762755 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.762637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-tmp-dir\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.762755 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.762675    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-home\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.764471 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.764448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-dshm\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.764720 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.764703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f07a290-4c91-464b-a84a-1c1e154da3c0-tls-certs\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.771549 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.771522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpnm4\" (UniqueName: \"kubernetes.io/projected/8f07a290-4c91-464b-a84a-1c1e154da3c0-kube-api-access-bpnm4\") pod \"ipp-route-264b08-kserve-5b6f6c46bc-78wgv\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") " pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:25.853507 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:25.853471    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:26.128097 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:26.128072    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"]
Sep 11 13:56:26.129619 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:56:26.129590    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f07a290_4c91_464b_a84a_1c1e154da3c0.slice/crio-056ecf1b43f791af9bc91717c7a8ba1e9cc5d134a8b0f92e50ed86ca603138eb WatchSource:0}: Error finding container 056ecf1b43f791af9bc91717c7a8ba1e9cc5d134a8b0f92e50ed86ca603138eb: Status 404 returned error can't find the container with id 056ecf1b43f791af9bc91717c7a8ba1e9cc5d134a8b0f92e50ed86ca603138eb
Sep 11 13:56:26.287627 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:26.287587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv" event={"ID":"8f07a290-4c91-464b-a84a-1c1e154da3c0","Type":"ContainerStarted","Data":"ce5b0c6507aac68d003d85e4f283a0852758978688e6bc1136d47f5039010a46"}
Sep 11 13:56:26.287627 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:26.287634    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv" event={"ID":"8f07a290-4c91-464b-a84a-1c1e154da3c0","Type":"ContainerStarted","Data":"056ecf1b43f791af9bc91717c7a8ba1e9cc5d134a8b0f92e50ed86ca603138eb"}
Sep 11 13:56:26.323968 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:26.323886    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv" podStartSLOduration=1.323870648 podStartE2EDuration="1.323870648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:56:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:56:26.323313854 +0000 UTC m=+2105.541851346" watchObservedRunningTime="2026-09-11 13:56:26.323870648 +0000 UTC m=+2105.542408144"
Sep 11 13:56:28.867730 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:28.867689    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:56:35.854109 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:35.854049    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:35.854109 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:35.854119    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:35.866848 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:35.866818    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:36.353178 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:36.353147    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:56:51.255274 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:51.255235    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:56:57.342312 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:57.342278    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc_07e30bc5-9bc0-47ab-b4e3-23dc72d19285/payload-processing/0.log"
Sep 11 13:56:57.616552 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:56:57.616457    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7_9326e58a-a408-4a5d-8e3a-6116eb3a2381/payload-processing/0.log"
Sep 11 13:57:06.329540 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:06.329505    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"]
Sep 11 13:57:06.333289 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:06.333274    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"
Sep 11 13:57:06.336532 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:06.336506    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-7b7819\""
Sep 11 13:57:06.336665 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:06.336506    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-7b7819-dockercfg-2xsrm\""
Sep 11 13:57:06.345524 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:06.345500    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"]
Sep 11 13:57:06.445487 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:06.445446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1edc9e0d-1fdf-4b50-8733-e0659740f56f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx\" (UID: \"1edc9e0d-1fdf-4b50-8733-e0659740f56f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"
Sep 11 13:57:06.445659 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:06.445600    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc926\" (UniqueName: \"kubernetes.io/projected/1edc9e0d-1fdf-4b50-8733-e0659740f56f-kube-api-access-zc926\") pod \"payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx\" (UID: \"1edc9e0d-1fdf-4b50-8733-e0659740f56f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"
Sep 11 13:57:06.546895 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:06.546844    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1edc9e0d-1fdf-4b50-8733-e0659740f56f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx\" (UID: \"1edc9e0d-1fdf-4b50-8733-e0659740f56f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"
Sep 11 13:57:06.547101 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:06.546972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc926\" (UniqueName: \"kubernetes.io/projected/1edc9e0d-1fdf-4b50-8733-e0659740f56f-kube-api-access-zc926\") pod \"payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx\" (UID: \"1edc9e0d-1fdf-4b50-8733-e0659740f56f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"
Sep 11 13:57:06.547594 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:06.547574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1edc9e0d-1fdf-4b50-8733-e0659740f56f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx\" (UID: \"1edc9e0d-1fdf-4b50-8733-e0659740f56f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"
Sep 11 13:57:06.557797 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:06.557774    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc926\" (UniqueName: \"kubernetes.io/projected/1edc9e0d-1fdf-4b50-8733-e0659740f56f-kube-api-access-zc926\") pod \"payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx\" (UID: \"1edc9e0d-1fdf-4b50-8733-e0659740f56f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"
Sep 11 13:57:06.645030 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:06.644941    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"
Sep 11 13:57:06.925882 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:57:06.925855    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1edc9e0d_1fdf_4b50_8733_e0659740f56f.slice/crio-241c72dfc58d69cf9191023879e6159cc82e595ea7cb81d57451e66af331a035 WatchSource:0}: Error finding container 241c72dfc58d69cf9191023879e6159cc82e595ea7cb81d57451e66af331a035: Status 404 returned error can't find the container with id 241c72dfc58d69cf9191023879e6159cc82e595ea7cb81d57451e66af331a035
Sep 11 13:57:06.926241 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:06.926214    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"]
Sep 11 13:57:07.518664 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:07.518632    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx" event={"ID":"1edc9e0d-1fdf-4b50-8733-e0659740f56f","Type":"ContainerStarted","Data":"e2c0e78a2b57f3b499da08309270b37e44c263be5a8a3280835b206c6e6eed6a"}
Sep 11 13:57:07.519115 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:07.518683    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx" event={"ID":"1edc9e0d-1fdf-4b50-8733-e0659740f56f","Type":"ContainerStarted","Data":"241c72dfc58d69cf9191023879e6159cc82e595ea7cb81d57451e66af331a035"}
Sep 11 13:57:07.519115 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:07.518821    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"
Sep 11 13:57:12.956582 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:12.956508    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx" podStartSLOduration=6.956483328 podStartE2EDuration="6.956483328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:57:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:07.546982671 +0000 UTC m=+2146.765520164" watchObservedRunningTime="2026-09-11 13:57:12.956483328 +0000 UTC m=+2152.175020821"
Sep 11 13:57:12.963119 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:12.963078    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:57:18.526919 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:18.526873    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"
Sep 11 13:57:21.824030 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:21.823998    2580 scope.go:117] "RemoveContainer" containerID="a268c187e2f233245bfc256bcf67ae2ac171e7ed9e14d667299914cca541347d"
Sep 11 13:57:43.095013 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.094973    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"]
Sep 11 13:57:43.095617 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.095253    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx" podUID="1edc9e0d-1fdf-4b50-8733-e0659740f56f" containerName="payload-processing" containerID="cri-o://e2c0e78a2b57f3b499da08309270b37e44c263be5a8a3280835b206c6e6eed6a" gracePeriod=30
Sep 11 13:57:43.351928 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.351859    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"
Sep 11 13:57:43.483216 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.483176    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1edc9e0d-1fdf-4b50-8733-e0659740f56f-plugins-config-volume\") pod \"1edc9e0d-1fdf-4b50-8733-e0659740f56f\" (UID: \"1edc9e0d-1fdf-4b50-8733-e0659740f56f\") "
Sep 11 13:57:43.483216 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.483222    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zc926\" (UniqueName: \"kubernetes.io/projected/1edc9e0d-1fdf-4b50-8733-e0659740f56f-kube-api-access-zc926\") pod \"1edc9e0d-1fdf-4b50-8733-e0659740f56f\" (UID: \"1edc9e0d-1fdf-4b50-8733-e0659740f56f\") "
Sep 11 13:57:43.484024 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.483980    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1edc9e0d-1fdf-4b50-8733-e0659740f56f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1edc9e0d-1fdf-4b50-8733-e0659740f56f" (UID: "1edc9e0d-1fdf-4b50-8733-e0659740f56f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:57:43.484635 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.484607    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1edc9e0d-1fdf-4b50-8733-e0659740f56f-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:57:43.488945 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.487038    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1edc9e0d-1fdf-4b50-8733-e0659740f56f-kube-api-access-zc926" (OuterVolumeSpecName: "kube-api-access-zc926") pod "1edc9e0d-1fdf-4b50-8733-e0659740f56f" (UID: "1edc9e0d-1fdf-4b50-8733-e0659740f56f"). InnerVolumeSpecName "kube-api-access-zc926". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:57:43.586110 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.586067    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc926\" (UniqueName: \"kubernetes.io/projected/1edc9e0d-1fdf-4b50-8733-e0659740f56f-kube-api-access-zc926\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:57:43.724157 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.724069    2580 generic.go:356] "Generic (PLEG): container finished" podID="1edc9e0d-1fdf-4b50-8733-e0659740f56f" containerID="e2c0e78a2b57f3b499da08309270b37e44c263be5a8a3280835b206c6e6eed6a" exitCode=0
Sep 11 13:57:43.724157 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.724111    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx" event={"ID":"1edc9e0d-1fdf-4b50-8733-e0659740f56f","Type":"ContainerDied","Data":"e2c0e78a2b57f3b499da08309270b37e44c263be5a8a3280835b206c6e6eed6a"}
Sep 11 13:57:43.724157 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.724120    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"
Sep 11 13:57:43.724383 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.724188    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx" event={"ID":"1edc9e0d-1fdf-4b50-8733-e0659740f56f","Type":"ContainerDied","Data":"241c72dfc58d69cf9191023879e6159cc82e595ea7cb81d57451e66af331a035"}
Sep 11 13:57:43.724383 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.724211    2580 scope.go:117] "RemoveContainer" containerID="e2c0e78a2b57f3b499da08309270b37e44c263be5a8a3280835b206c6e6eed6a"
Sep 11 13:57:43.733942 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.733920    2580 scope.go:117] "RemoveContainer" containerID="e2c0e78a2b57f3b499da08309270b37e44c263be5a8a3280835b206c6e6eed6a"
Sep 11 13:57:43.734181 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:57:43.734162    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2c0e78a2b57f3b499da08309270b37e44c263be5a8a3280835b206c6e6eed6a\": container with ID starting with e2c0e78a2b57f3b499da08309270b37e44c263be5a8a3280835b206c6e6eed6a not found: ID does not exist" containerID="e2c0e78a2b57f3b499da08309270b37e44c263be5a8a3280835b206c6e6eed6a"
Sep 11 13:57:43.734241 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.734189    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2c0e78a2b57f3b499da08309270b37e44c263be5a8a3280835b206c6e6eed6a"} err="failed to get container status \"e2c0e78a2b57f3b499da08309270b37e44c263be5a8a3280835b206c6e6eed6a\": rpc error: code = NotFound desc = could not find container \"e2c0e78a2b57f3b499da08309270b37e44c263be5a8a3280835b206c6e6eed6a\": container with ID starting with e2c0e78a2b57f3b499da08309270b37e44c263be5a8a3280835b206c6e6eed6a not found: ID does not exist"
Sep 11 13:57:43.747222 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.747197    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"]
Sep 11 13:57:43.751394 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:43.751371    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-7b7819-5c646444bc-fqcnx"]
Sep 11 13:57:45.316039 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:45.316007    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1edc9e0d-1fdf-4b50-8733-e0659740f56f" path="/var/lib/kubelet/pods/1edc9e0d-1fdf-4b50-8733-e0659740f56f/volumes"
Sep 11 13:57:48.351189 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:57:48.351153    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:58:07.457754 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:07.457712    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:58:09.981589 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:09.981551    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"]
Sep 11 13:58:09.982097 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:09.981788    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc" podUID="07e30bc5-9bc0-47ab-b4e3-23dc72d19285" containerName="payload-processing" containerID="cri-o://a749b9922daa50ab61a88d6b971ed3a708fa9098e2cff5238de94461f123a86d" gracePeriod=30
Sep 11 13:58:10.242145 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.242087    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"
Sep 11 13:58:10.332127 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.332092    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07e30bc5-9bc0-47ab-b4e3-23dc72d19285-plugins-config-volume\") pod \"07e30bc5-9bc0-47ab-b4e3-23dc72d19285\" (UID: \"07e30bc5-9bc0-47ab-b4e3-23dc72d19285\") "
Sep 11 13:58:10.332305 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.332169    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6wqh\" (UniqueName: \"kubernetes.io/projected/07e30bc5-9bc0-47ab-b4e3-23dc72d19285-kube-api-access-v6wqh\") pod \"07e30bc5-9bc0-47ab-b4e3-23dc72d19285\" (UID: \"07e30bc5-9bc0-47ab-b4e3-23dc72d19285\") "
Sep 11 13:58:10.332476 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.332454    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07e30bc5-9bc0-47ab-b4e3-23dc72d19285-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "07e30bc5-9bc0-47ab-b4e3-23dc72d19285" (UID: "07e30bc5-9bc0-47ab-b4e3-23dc72d19285"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:58:10.334288 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.334260    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07e30bc5-9bc0-47ab-b4e3-23dc72d19285-kube-api-access-v6wqh" (OuterVolumeSpecName: "kube-api-access-v6wqh") pod "07e30bc5-9bc0-47ab-b4e3-23dc72d19285" (UID: "07e30bc5-9bc0-47ab-b4e3-23dc72d19285"). InnerVolumeSpecName "kube-api-access-v6wqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:58:10.433695 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.433657    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07e30bc5-9bc0-47ab-b4e3-23dc72d19285-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:58:10.433695 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.433688    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6wqh\" (UniqueName: \"kubernetes.io/projected/07e30bc5-9bc0-47ab-b4e3-23dc72d19285-kube-api-access-v6wqh\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:58:10.883485 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.883450    2580 generic.go:356] "Generic (PLEG): container finished" podID="07e30bc5-9bc0-47ab-b4e3-23dc72d19285" containerID="a749b9922daa50ab61a88d6b971ed3a708fa9098e2cff5238de94461f123a86d" exitCode=0
Sep 11 13:58:10.883672 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.883496    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"
Sep 11 13:58:10.883672 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.883536    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc" event={"ID":"07e30bc5-9bc0-47ab-b4e3-23dc72d19285","Type":"ContainerDied","Data":"a749b9922daa50ab61a88d6b971ed3a708fa9098e2cff5238de94461f123a86d"}
Sep 11 13:58:10.883672 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.883592    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc" event={"ID":"07e30bc5-9bc0-47ab-b4e3-23dc72d19285","Type":"ContainerDied","Data":"bf513ceb9dcb4c38f5610bc00cb5c9ab2a2c464a581d26def660fe6f0d576843"}
Sep 11 13:58:10.883672 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.883619    2580 scope.go:117] "RemoveContainer" containerID="a749b9922daa50ab61a88d6b971ed3a708fa9098e2cff5238de94461f123a86d"
Sep 11 13:58:10.893503 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.893486    2580 scope.go:117] "RemoveContainer" containerID="a749b9922daa50ab61a88d6b971ed3a708fa9098e2cff5238de94461f123a86d"
Sep 11 13:58:10.893770 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:58:10.893752    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a749b9922daa50ab61a88d6b971ed3a708fa9098e2cff5238de94461f123a86d\": container with ID starting with a749b9922daa50ab61a88d6b971ed3a708fa9098e2cff5238de94461f123a86d not found: ID does not exist" containerID="a749b9922daa50ab61a88d6b971ed3a708fa9098e2cff5238de94461f123a86d"
Sep 11 13:58:10.893852 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.893777    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a749b9922daa50ab61a88d6b971ed3a708fa9098e2cff5238de94461f123a86d"} err="failed to get container status \"a749b9922daa50ab61a88d6b971ed3a708fa9098e2cff5238de94461f123a86d\": rpc error: code = NotFound desc = could not find container \"a749b9922daa50ab61a88d6b971ed3a708fa9098e2cff5238de94461f123a86d\": container with ID starting with a749b9922daa50ab61a88d6b971ed3a708fa9098e2cff5238de94461f123a86d not found: ID does not exist"
Sep 11 13:58:10.910450 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.910427    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"]
Sep 11 13:58:10.912731 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:10.912711    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-264b08-77575c5557-hg7jc"]
Sep 11 13:58:11.316339 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:11.316305    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07e30bc5-9bc0-47ab-b4e3-23dc72d19285" path="/var/lib/kubelet/pods/07e30bc5-9bc0-47ab-b4e3-23dc72d19285/volumes"
Sep 11 13:58:15.581145 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:15.581109    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"]
Sep 11 13:58:20.663296 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:20.663260    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:58:30.600706 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:30.600659    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv" podUID="8f07a290-4c91-464b-a84a-1c1e154da3c0" containerName="main" containerID="cri-o://ce5b0c6507aac68d003d85e4f283a0852758978688e6bc1136d47f5039010a46" gracePeriod=45
Sep 11 13:58:30.845475 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:30.845450    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:58:30.916751 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:30.916673    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-home\") pod \"8f07a290-4c91-464b-a84a-1c1e154da3c0\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") "
Sep 11 13:58:30.916751 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:30.916713    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-tmp-dir\") pod \"8f07a290-4c91-464b-a84a-1c1e154da3c0\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") "
Sep 11 13:58:30.917019 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:30.916766    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f07a290-4c91-464b-a84a-1c1e154da3c0-tls-certs\") pod \"8f07a290-4c91-464b-a84a-1c1e154da3c0\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") "
Sep 11 13:58:30.917019 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:30.916784    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-model-cache\") pod \"8f07a290-4c91-464b-a84a-1c1e154da3c0\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") "
Sep 11 13:58:30.917135 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:30.917016    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-home" (OuterVolumeSpecName: "home") pod "8f07a290-4c91-464b-a84a-1c1e154da3c0" (UID: "8f07a290-4c91-464b-a84a-1c1e154da3c0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:58:30.917135 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:30.917103    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8f07a290-4c91-464b-a84a-1c1e154da3c0" (UID: "8f07a290-4c91-464b-a84a-1c1e154da3c0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:58:30.917135 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:30.917113    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-model-cache" (OuterVolumeSpecName: "model-cache") pod "8f07a290-4c91-464b-a84a-1c1e154da3c0" (UID: "8f07a290-4c91-464b-a84a-1c1e154da3c0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:58:30.919065 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:30.919041    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f07a290-4c91-464b-a84a-1c1e154da3c0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8f07a290-4c91-464b-a84a-1c1e154da3c0" (UID: "8f07a290-4c91-464b-a84a-1c1e154da3c0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:58:30.997560 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:30.997523    2580 generic.go:356] "Generic (PLEG): container finished" podID="8f07a290-4c91-464b-a84a-1c1e154da3c0" containerID="ce5b0c6507aac68d003d85e4f283a0852758978688e6bc1136d47f5039010a46" exitCode=0
Sep 11 13:58:30.997734 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:30.997567    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"
Sep 11 13:58:30.997734 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:30.997610    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv" event={"ID":"8f07a290-4c91-464b-a84a-1c1e154da3c0","Type":"ContainerDied","Data":"ce5b0c6507aac68d003d85e4f283a0852758978688e6bc1136d47f5039010a46"}
Sep 11 13:58:30.997734 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:30.997654    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv" event={"ID":"8f07a290-4c91-464b-a84a-1c1e154da3c0","Type":"ContainerDied","Data":"056ecf1b43f791af9bc91717c7a8ba1e9cc5d134a8b0f92e50ed86ca603138eb"}
Sep 11 13:58:30.997734 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:30.997678    2580 scope.go:117] "RemoveContainer" containerID="ce5b0c6507aac68d003d85e4f283a0852758978688e6bc1136d47f5039010a46"
Sep 11 13:58:31.006402 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:31.006382    2580 scope.go:117] "RemoveContainer" containerID="ce5b0c6507aac68d003d85e4f283a0852758978688e6bc1136d47f5039010a46"
Sep 11 13:58:31.006727 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:58:31.006706    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce5b0c6507aac68d003d85e4f283a0852758978688e6bc1136d47f5039010a46\": container with ID starting with ce5b0c6507aac68d003d85e4f283a0852758978688e6bc1136d47f5039010a46 not found: ID does not exist" containerID="ce5b0c6507aac68d003d85e4f283a0852758978688e6bc1136d47f5039010a46"
Sep 11 13:58:31.006782 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:31.006739    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce5b0c6507aac68d003d85e4f283a0852758978688e6bc1136d47f5039010a46"} err="failed to get container status \"ce5b0c6507aac68d003d85e4f283a0852758978688e6bc1136d47f5039010a46\": rpc error: code = NotFound desc = could not find container \"ce5b0c6507aac68d003d85e4f283a0852758978688e6bc1136d47f5039010a46\": container with ID starting with ce5b0c6507aac68d003d85e4f283a0852758978688e6bc1136d47f5039010a46 not found: ID does not exist"
Sep 11 13:58:31.018193 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:31.018169    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpnm4\" (UniqueName: \"kubernetes.io/projected/8f07a290-4c91-464b-a84a-1c1e154da3c0-kube-api-access-bpnm4\") pod \"8f07a290-4c91-464b-a84a-1c1e154da3c0\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") "
Sep 11 13:58:31.018364 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:31.018204    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-dshm\") pod \"8f07a290-4c91-464b-a84a-1c1e154da3c0\" (UID: \"8f07a290-4c91-464b-a84a-1c1e154da3c0\") "
Sep 11 13:58:31.018364 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:31.018364    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:58:31.018481 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:31.018374    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:58:31.018481 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:31.018383    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f07a290-4c91-464b-a84a-1c1e154da3c0-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:58:31.018481 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:31.018390    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:58:31.020285 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:31.020256    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-dshm" (OuterVolumeSpecName: "dshm") pod "8f07a290-4c91-464b-a84a-1c1e154da3c0" (UID: "8f07a290-4c91-464b-a84a-1c1e154da3c0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:58:31.020386 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:31.020348    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f07a290-4c91-464b-a84a-1c1e154da3c0-kube-api-access-bpnm4" (OuterVolumeSpecName: "kube-api-access-bpnm4") pod "8f07a290-4c91-464b-a84a-1c1e154da3c0" (UID: "8f07a290-4c91-464b-a84a-1c1e154da3c0"). InnerVolumeSpecName "kube-api-access-bpnm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:58:31.119061 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:31.119028    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpnm4\" (UniqueName: \"kubernetes.io/projected/8f07a290-4c91-464b-a84a-1c1e154da3c0-kube-api-access-bpnm4\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:58:31.119061 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:31.119056    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f07a290-4c91-464b-a84a-1c1e154da3c0-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:58:31.321201 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:31.321170    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"]
Sep 11 13:58:31.326411 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:31.326384    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-264b08/ipp-route-264b08-kserve-5b6f6c46bc-78wgv"]
Sep 11 13:58:33.316084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:33.316047    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f07a290-4c91-464b-a84a-1c1e154da3c0" path="/var/lib/kubelet/pods/8f07a290-4c91-464b-a84a-1c1e154da3c0/volumes"
Sep 11 13:58:55.915489 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:55.915448    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx"]
Sep 11 13:58:55.916868 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:55.916836    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07e30bc5-9bc0-47ab-b4e3-23dc72d19285" containerName="payload-processing"
Sep 11 13:58:55.917034 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:55.917020    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="07e30bc5-9bc0-47ab-b4e3-23dc72d19285" containerName="payload-processing"
Sep 11 13:58:55.917144 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:55.917133    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1edc9e0d-1fdf-4b50-8733-e0659740f56f" containerName="payload-processing"
Sep 11 13:58:55.917216 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:55.917207    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1edc9e0d-1fdf-4b50-8733-e0659740f56f" containerName="payload-processing"
Sep 11 13:58:55.917337 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:55.917327    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f07a290-4c91-464b-a84a-1c1e154da3c0" containerName="main"
Sep 11 13:58:55.917412 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:55.917404    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f07a290-4c91-464b-a84a-1c1e154da3c0" containerName="main"
Sep 11 13:58:55.917676 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:55.917663    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1edc9e0d-1fdf-4b50-8733-e0659740f56f" containerName="payload-processing"
Sep 11 13:58:55.917769 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:55.917759    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="07e30bc5-9bc0-47ab-b4e3-23dc72d19285" containerName="payload-processing"
Sep 11 13:58:55.917840 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:55.917832    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f07a290-4c91-464b-a84a-1c1e154da3c0" containerName="main"
Sep 11 13:58:55.923510 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:55.923487    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx"
Sep 11 13:58:55.926118 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:58:55.926093    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" podUID="d12d24f2-c73a-4042-b90f-925af0c2334b" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx"
Sep 11 13:58:55.937843 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:58:55.937816    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-142-132.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-142-132.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 11 13:58:55.938052 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:55.938040    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:58:55.963701 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:55.963677    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx"]
Sep 11 13:58:56.058850 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:56.058788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d12d24f2-c73a-4042-b90f-925af0c2334b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx\" (UID: \"d12d24f2-c73a-4042-b90f-925af0c2334b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx"
Sep 11 13:58:56.160049 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:56.160013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d12d24f2-c73a-4042-b90f-925af0c2334b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx\" (UID: \"d12d24f2-c73a-4042-b90f-925af0c2334b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx"
Sep 11 13:58:56.770365 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:56.770331    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:58:56.781139 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:56.781113    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d12d24f2-c73a-4042-b90f-925af0c2334b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx\" (UID: \"d12d24f2-c73a-4042-b90f-925af0c2334b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx"
Sep 11 13:58:56.834137 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:56.834102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx"
Sep 11 13:58:57.103431 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:57.103403    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx"]
Sep 11 13:58:57.105043 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:58:57.105001    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd12d24f2_c73a_4042_b90f_925af0c2334b.slice/crio-468c19eec3fc20934e7220aac4cb24cbafef0073c5694902c265933bb4c2f53a WatchSource:0}: Error finding container 468c19eec3fc20934e7220aac4cb24cbafef0073c5694902c265933bb4c2f53a: Status 404 returned error can't find the container with id 468c19eec3fc20934e7220aac4cb24cbafef0073c5694902c265933bb4c2f53a
Sep 11 13:58:57.144501 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:57.144468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx" event={"ID":"d12d24f2-c73a-4042-b90f-925af0c2334b","Type":"ContainerStarted","Data":"468c19eec3fc20934e7220aac4cb24cbafef0073c5694902c265933bb4c2f53a"}
Sep 11 13:58:58.151268 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:58.151237    2580 generic.go:356] "Generic (PLEG): container finished" podID="d12d24f2-c73a-4042-b90f-925af0c2334b" containerID="f860f8503d65d422bd30d990db639b2255d2935a672ad6c371d8332f53cc86ec" exitCode=0
Sep 11 13:58:58.151641 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:58.151322    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx" event={"ID":"d12d24f2-c73a-4042-b90f-925af0c2334b","Type":"ContainerDied","Data":"f860f8503d65d422bd30d990db639b2255d2935a672ad6c371d8332f53cc86ec"}
Sep 11 13:58:59.289892 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:59.289869    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx"
Sep 11 13:58:59.385772 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:59.385735    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d12d24f2-c73a-4042-b90f-925af0c2334b-maas-api-service-ca\") pod \"d12d24f2-c73a-4042-b90f-925af0c2334b\" (UID: \"d12d24f2-c73a-4042-b90f-925af0c2334b\") "
Sep 11 13:58:59.386156 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:59.386136    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d12d24f2-c73a-4042-b90f-925af0c2334b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d12d24f2-c73a-4042-b90f-925af0c2334b" (UID: "d12d24f2-c73a-4042-b90f-925af0c2334b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:58:59.487211 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:58:59.487125    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d12d24f2-c73a-4042-b90f-925af0c2334b-maas-api-service-ca\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:59:00.164044 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:00.164016    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx"
Sep 11 13:59:00.164218 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:00.164007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx" event={"ID":"d12d24f2-c73a-4042-b90f-925af0c2334b","Type":"ContainerDied","Data":"468c19eec3fc20934e7220aac4cb24cbafef0073c5694902c265933bb4c2f53a"}
Sep 11 13:59:00.164218 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:00.164152    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="468c19eec3fc20934e7220aac4cb24cbafef0073c5694902c265933bb4c2f53a"
Sep 11 13:59:06.918161 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:06.918125    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"]
Sep 11 13:59:06.918710 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:06.918384    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7" podUID="9326e58a-a408-4a5d-8e3a-6116eb3a2381" containerName="payload-processing" containerID="cri-o://f27805378ed7b11c794743e3ce8bd474bcedc1c852a0e6d28de18ddca3fc64d6" gracePeriod=30
Sep 11 13:59:06.960331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:06.960289    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"]
Sep 11 13:59:06.960539 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:06.960517    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt" podUID="55b268b7-44da-4f02-823e-b60eb2eb3c03" containerName="payload-pre-processing" containerID="cri-o://412d5f809d6b1656e04812f3af79bc4d77835f04b4378acf3ae668f353d6a1d9" gracePeriod=30
Sep 11 13:59:07.204571 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.204483    2580 generic.go:356] "Generic (PLEG): container finished" podID="9326e58a-a408-4a5d-8e3a-6116eb3a2381" containerID="f27805378ed7b11c794743e3ce8bd474bcedc1c852a0e6d28de18ddca3fc64d6" exitCode=0
Sep 11 13:59:07.204816 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.204567    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7" event={"ID":"9326e58a-a408-4a5d-8e3a-6116eb3a2381","Type":"ContainerDied","Data":"f27805378ed7b11c794743e3ce8bd474bcedc1c852a0e6d28de18ddca3fc64d6"}
Sep 11 13:59:07.206539 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.206472    2580 generic.go:356] "Generic (PLEG): container finished" podID="55b268b7-44da-4f02-823e-b60eb2eb3c03" containerID="412d5f809d6b1656e04812f3af79bc4d77835f04b4378acf3ae668f353d6a1d9" exitCode=0
Sep 11 13:59:07.206658 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.206536    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt" event={"ID":"55b268b7-44da-4f02-823e-b60eb2eb3c03","Type":"ContainerDied","Data":"412d5f809d6b1656e04812f3af79bc4d77835f04b4378acf3ae668f353d6a1d9"}
Sep 11 13:59:07.233108 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.233089    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"
Sep 11 13:59:07.236013 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.235996    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"
Sep 11 13:59:07.358943 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.358895    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx"]
Sep 11 13:59:07.359474 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.359457    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55b268b7-44da-4f02-823e-b60eb2eb3c03-plugins-config-volume\") pod \"55b268b7-44da-4f02-823e-b60eb2eb3c03\" (UID: \"55b268b7-44da-4f02-823e-b60eb2eb3c03\") "
Sep 11 13:59:07.359526 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.359517    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crtgs\" (UniqueName: \"kubernetes.io/projected/9326e58a-a408-4a5d-8e3a-6116eb3a2381-kube-api-access-crtgs\") pod \"9326e58a-a408-4a5d-8e3a-6116eb3a2381\" (UID: \"9326e58a-a408-4a5d-8e3a-6116eb3a2381\") "
Sep 11 13:59:07.359569 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.359547    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qc56p\" (UniqueName: \"kubernetes.io/projected/55b268b7-44da-4f02-823e-b60eb2eb3c03-kube-api-access-qc56p\") pod \"55b268b7-44da-4f02-823e-b60eb2eb3c03\" (UID: \"55b268b7-44da-4f02-823e-b60eb2eb3c03\") "
Sep 11 13:59:07.359611 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.359588    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9326e58a-a408-4a5d-8e3a-6116eb3a2381-plugins-config-volume\") pod \"9326e58a-a408-4a5d-8e3a-6116eb3a2381\" (UID: \"9326e58a-a408-4a5d-8e3a-6116eb3a2381\") "
Sep 11 13:59:07.359945 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.359884    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55b268b7-44da-4f02-823e-b60eb2eb3c03-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "55b268b7-44da-4f02-823e-b60eb2eb3c03" (UID: "55b268b7-44da-4f02-823e-b60eb2eb3c03"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:59:07.360098 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.360004    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9326e58a-a408-4a5d-8e3a-6116eb3a2381-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9326e58a-a408-4a5d-8e3a-6116eb3a2381" (UID: "9326e58a-a408-4a5d-8e3a-6116eb3a2381"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:59:07.361689 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.361668    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55b268b7-44da-4f02-823e-b60eb2eb3c03-kube-api-access-qc56p" (OuterVolumeSpecName: "kube-api-access-qc56p") pod "55b268b7-44da-4f02-823e-b60eb2eb3c03" (UID: "55b268b7-44da-4f02-823e-b60eb2eb3c03"). InnerVolumeSpecName "kube-api-access-qc56p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:59:07.361774 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.361695    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9326e58a-a408-4a5d-8e3a-6116eb3a2381-kube-api-access-crtgs" (OuterVolumeSpecName: "kube-api-access-crtgs") pod "9326e58a-a408-4a5d-8e3a-6116eb3a2381" (UID: "9326e58a-a408-4a5d-8e3a-6116eb3a2381"). InnerVolumeSpecName "kube-api-access-crtgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:59:07.365409 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.365385    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a403bc-gxgxx"]
Sep 11 13:59:07.461107 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.461023    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55b268b7-44da-4f02-823e-b60eb2eb3c03-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:59:07.461107 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.461053    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crtgs\" (UniqueName: \"kubernetes.io/projected/9326e58a-a408-4a5d-8e3a-6116eb3a2381-kube-api-access-crtgs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:59:07.461107 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.461063    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qc56p\" (UniqueName: \"kubernetes.io/projected/55b268b7-44da-4f02-823e-b60eb2eb3c03-kube-api-access-qc56p\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:59:07.461107 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:07.461074    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9326e58a-a408-4a5d-8e3a-6116eb3a2381-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:59:08.213552 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:08.213522    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"
Sep 11 13:59:08.214018 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:08.213550    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7" event={"ID":"9326e58a-a408-4a5d-8e3a-6116eb3a2381","Type":"ContainerDied","Data":"6a0e80f01f903bbc657a320b93a2cd319515d1ed4ceabec1b59c2dceeb3014cf"}
Sep 11 13:59:08.214018 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:08.213618    2580 scope.go:117] "RemoveContainer" containerID="f27805378ed7b11c794743e3ce8bd474bcedc1c852a0e6d28de18ddca3fc64d6"
Sep 11 13:59:08.214969 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:08.214952    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"
Sep 11 13:59:08.215112 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:08.214970    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt" event={"ID":"55b268b7-44da-4f02-823e-b60eb2eb3c03","Type":"ContainerDied","Data":"21972e0ec77634e627d1048f1996c2cc3725b46404b25e6075b940d0a8134a2b"}
Sep 11 13:59:08.223781 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:08.223761    2580 scope.go:117] "RemoveContainer" containerID="412d5f809d6b1656e04812f3af79bc4d77835f04b4378acf3ae668f353d6a1d9"
Sep 11 13:59:08.260968 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:08.260939    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"]
Sep 11 13:59:08.275743 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:08.275712    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a403bc-6859b89bd5-55jr7"]
Sep 11 13:59:08.305892 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:08.305857    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"]
Sep 11 13:59:08.310699 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:08.310675    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a403bc-86867c94db-wfjpt"]
Sep 11 13:59:09.316040 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:09.316008    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55b268b7-44da-4f02-823e-b60eb2eb3c03" path="/var/lib/kubelet/pods/55b268b7-44da-4f02-823e-b60eb2eb3c03/volumes"
Sep 11 13:59:09.316432 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:09.316380    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9326e58a-a408-4a5d-8e3a-6116eb3a2381" path="/var/lib/kubelet/pods/9326e58a-a408-4a5d-8e3a-6116eb3a2381/volumes"
Sep 11 13:59:09.316668 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:09.316655    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d12d24f2-c73a-4042-b90f-925af0c2334b" path="/var/lib/kubelet/pods/d12d24f2-c73a-4042-b90f-925af0c2334b/volumes"
Sep 11 13:59:10.310463 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:10.310430    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:59:15.166462 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.166425    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5"]
Sep 11 13:59:15.166838 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.166825    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9326e58a-a408-4a5d-8e3a-6116eb3a2381" containerName="payload-processing"
Sep 11 13:59:15.166894 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.166840    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9326e58a-a408-4a5d-8e3a-6116eb3a2381" containerName="payload-processing"
Sep 11 13:59:15.166894 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.166856    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55b268b7-44da-4f02-823e-b60eb2eb3c03" containerName="payload-pre-processing"
Sep 11 13:59:15.166894 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.166862    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="55b268b7-44da-4f02-823e-b60eb2eb3c03" containerName="payload-pre-processing"
Sep 11 13:59:15.166894 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.166870    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d12d24f2-c73a-4042-b90f-925af0c2334b" containerName="revoke-keys"
Sep 11 13:59:15.166894 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.166876    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d12d24f2-c73a-4042-b90f-925af0c2334b" containerName="revoke-keys"
Sep 11 13:59:15.167078 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.166950    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9326e58a-a408-4a5d-8e3a-6116eb3a2381" containerName="payload-processing"
Sep 11 13:59:15.167078 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.166959    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="55b268b7-44da-4f02-823e-b60eb2eb3c03" containerName="payload-pre-processing"
Sep 11 13:59:15.167078 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.166966    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d12d24f2-c73a-4042-b90f-925af0c2334b" containerName="revoke-keys"
Sep 11 13:59:15.171331 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.171315    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5"
Sep 11 13:59:15.174014 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.173985    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:59:15.174140 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.174052    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:59:15.179956 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.179886    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5"]
Sep 11 13:59:15.222627 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.222589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/056623be-e3cc-444e-afd7-c7238962c233-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5\" (UID: \"056623be-e3cc-444e-afd7-c7238962c233\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5"
Sep 11 13:59:15.323459 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.323403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/056623be-e3cc-444e-afd7-c7238962c233-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5\" (UID: \"056623be-e3cc-444e-afd7-c7238962c233\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5"
Sep 11 13:59:15.324084 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.324062    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/056623be-e3cc-444e-afd7-c7238962c233-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5\" (UID: \"056623be-e3cc-444e-afd7-c7238962c233\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5"
Sep 11 13:59:15.482728 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.482637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5"
Sep 11 13:59:15.766667 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:15.766636    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5"]
Sep 11 13:59:15.768239 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:59:15.768210    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod056623be_e3cc_444e_afd7_c7238962c233.slice/crio-792942ad922d7854cc9e807b10e1e2d64fe69b2101d7cc8324d50b4829b4bb17 WatchSource:0}: Error finding container 792942ad922d7854cc9e807b10e1e2d64fe69b2101d7cc8324d50b4829b4bb17: Status 404 returned error can't find the container with id 792942ad922d7854cc9e807b10e1e2d64fe69b2101d7cc8324d50b4829b4bb17
Sep 11 13:59:16.262980 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:16.262934    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5" event={"ID":"056623be-e3cc-444e-afd7-c7238962c233","Type":"ContainerStarted","Data":"1a24e1e65d6eccb03c3c57fba488d22839f4323bc991f985d47ece8dfd48285e"}
Sep 11 13:59:16.263359 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:16.263002    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5" event={"ID":"056623be-e3cc-444e-afd7-c7238962c233","Type":"ContainerStarted","Data":"792942ad922d7854cc9e807b10e1e2d64fe69b2101d7cc8324d50b4829b4bb17"}
Sep 11 13:59:16.283356 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:16.283310    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5" podStartSLOduration=1.283294922 podStartE2EDuration="1.283294922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:59:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:16.281898118 +0000 UTC m=+2275.500435610" watchObservedRunningTime="2026-09-11 13:59:16.283294922 +0000 UTC m=+2275.501832408"
Sep 11 13:59:17.269789 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:17.269754    2580 generic.go:356] "Generic (PLEG): container finished" podID="056623be-e3cc-444e-afd7-c7238962c233" containerID="1a24e1e65d6eccb03c3c57fba488d22839f4323bc991f985d47ece8dfd48285e" exitCode=0
Sep 11 13:59:17.270290 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:17.269851    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5" event={"ID":"056623be-e3cc-444e-afd7-c7238962c233","Type":"ContainerDied","Data":"1a24e1e65d6eccb03c3c57fba488d22839f4323bc991f985d47ece8dfd48285e"}
Sep 11 13:59:18.407823 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:18.407800    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5"
Sep 11 13:59:18.449305 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:18.449274    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/056623be-e3cc-444e-afd7-c7238962c233-maas-api-service-ca\") pod \"056623be-e3cc-444e-afd7-c7238962c233\" (UID: \"056623be-e3cc-444e-afd7-c7238962c233\") "
Sep 11 13:59:18.449650 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:18.449625    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/056623be-e3cc-444e-afd7-c7238962c233-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "056623be-e3cc-444e-afd7-c7238962c233" (UID: "056623be-e3cc-444e-afd7-c7238962c233"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:59:18.550971 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:18.550883    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/056623be-e3cc-444e-afd7-c7238962c233-maas-api-service-ca\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:59:19.282848 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:19.282815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5" event={"ID":"056623be-e3cc-444e-afd7-c7238962c233","Type":"ContainerDied","Data":"792942ad922d7854cc9e807b10e1e2d64fe69b2101d7cc8324d50b4829b4bb17"}
Sep 11 13:59:19.282848 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:19.282838    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5"
Sep 11 13:59:19.283079 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:19.282890    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="792942ad922d7854cc9e807b10e1e2d64fe69b2101d7cc8324d50b4829b4bb17"
Sep 11 13:59:25.850136 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:25.850099    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0317c3-29818905-frz76"]
Sep 11 13:59:25.852740 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:25.852715    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-0317c3-29818905-frz76"]
Sep 11 13:59:25.889290 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:25.889250    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"]
Sep 11 13:59:25.889562 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:25.889529    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz" podUID="e1da49c8-c4fe-411b-89e5-e08bbcfea7db" containerName="payload-processing" containerID="cri-o://2b2de0799a80548fd37e1a84f856424f8cbd7fe1a3fc7e1e50e457cf950ec2c9" gracePeriod=30
Sep 11 13:59:26.088469 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.088409    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5"]
Sep 11 13:59:26.093445 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.093409    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0317c3-dx4c5"]
Sep 11 13:59:26.141993 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.141969    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"
Sep 11 13:59:26.212461 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.212428    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1da49c8-c4fe-411b-89e5-e08bbcfea7db-plugins-config-volume\") pod \"e1da49c8-c4fe-411b-89e5-e08bbcfea7db\" (UID: \"e1da49c8-c4fe-411b-89e5-e08bbcfea7db\") "
Sep 11 13:59:26.212649 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.212483    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzcwr\" (UniqueName: \"kubernetes.io/projected/e1da49c8-c4fe-411b-89e5-e08bbcfea7db-kube-api-access-qzcwr\") pod \"e1da49c8-c4fe-411b-89e5-e08bbcfea7db\" (UID: \"e1da49c8-c4fe-411b-89e5-e08bbcfea7db\") "
Sep 11 13:59:26.212876 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.212851    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1da49c8-c4fe-411b-89e5-e08bbcfea7db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e1da49c8-c4fe-411b-89e5-e08bbcfea7db" (UID: "e1da49c8-c4fe-411b-89e5-e08bbcfea7db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:59:26.214596 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.214578    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1da49c8-c4fe-411b-89e5-e08bbcfea7db-kube-api-access-qzcwr" (OuterVolumeSpecName: "kube-api-access-qzcwr") pod "e1da49c8-c4fe-411b-89e5-e08bbcfea7db" (UID: "e1da49c8-c4fe-411b-89e5-e08bbcfea7db"). InnerVolumeSpecName "kube-api-access-qzcwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:59:26.313788 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.313755    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1da49c8-c4fe-411b-89e5-e08bbcfea7db-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:59:26.313788 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.313784    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzcwr\" (UniqueName: \"kubernetes.io/projected/e1da49c8-c4fe-411b-89e5-e08bbcfea7db-kube-api-access-qzcwr\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:59:26.323931 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.323879    2580 generic.go:356] "Generic (PLEG): container finished" podID="e1da49c8-c4fe-411b-89e5-e08bbcfea7db" containerID="2b2de0799a80548fd37e1a84f856424f8cbd7fe1a3fc7e1e50e457cf950ec2c9" exitCode=0
Sep 11 13:59:26.324092 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.323939    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"
Sep 11 13:59:26.324092 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.323945    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz" event={"ID":"e1da49c8-c4fe-411b-89e5-e08bbcfea7db","Type":"ContainerDied","Data":"2b2de0799a80548fd37e1a84f856424f8cbd7fe1a3fc7e1e50e457cf950ec2c9"}
Sep 11 13:59:26.324092 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.323993    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz" event={"ID":"e1da49c8-c4fe-411b-89e5-e08bbcfea7db","Type":"ContainerDied","Data":"f454c7ffc3552239ed66d73ec3f2dd1f03b1a6fbd781f32dca8b61126303da58"}
Sep 11 13:59:26.324092 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.324016    2580 scope.go:117] "RemoveContainer" containerID="2b2de0799a80548fd37e1a84f856424f8cbd7fe1a3fc7e1e50e457cf950ec2c9"
Sep 11 13:59:26.333153 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.333135    2580 scope.go:117] "RemoveContainer" containerID="2b2de0799a80548fd37e1a84f856424f8cbd7fe1a3fc7e1e50e457cf950ec2c9"
Sep 11 13:59:26.333425 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:59:26.333408    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b2de0799a80548fd37e1a84f856424f8cbd7fe1a3fc7e1e50e457cf950ec2c9\": container with ID starting with 2b2de0799a80548fd37e1a84f856424f8cbd7fe1a3fc7e1e50e457cf950ec2c9 not found: ID does not exist" containerID="2b2de0799a80548fd37e1a84f856424f8cbd7fe1a3fc7e1e50e457cf950ec2c9"
Sep 11 13:59:26.333464 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.333434    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b2de0799a80548fd37e1a84f856424f8cbd7fe1a3fc7e1e50e457cf950ec2c9"} err="failed to get container status \"2b2de0799a80548fd37e1a84f856424f8cbd7fe1a3fc7e1e50e457cf950ec2c9\": rpc error: code = NotFound desc = could not find container \"2b2de0799a80548fd37e1a84f856424f8cbd7fe1a3fc7e1e50e457cf950ec2c9\": container with ID starting with 2b2de0799a80548fd37e1a84f856424f8cbd7fe1a3fc7e1e50e457cf950ec2c9 not found: ID does not exist"
Sep 11 13:59:26.347495 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.347464    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"]
Sep 11 13:59:26.352376 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:26.352352    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0317c3-589b6cd8c7-5stjz"]
Sep 11 13:59:27.315252 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:27.315219    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="056623be-e3cc-444e-afd7-c7238962c233" path="/var/lib/kubelet/pods/056623be-e3cc-444e-afd7-c7238962c233/volumes"
Sep 11 13:59:27.315618 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:27.315519    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81916f38-81a9-4b6a-b13f-e0d19557883d" path="/var/lib/kubelet/pods/81916f38-81a9-4b6a-b13f-e0d19557883d/volumes"
Sep 11 13:59:27.315809 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:27.315797    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1da49c8-c4fe-411b-89e5-e08bbcfea7db" path="/var/lib/kubelet/pods/e1da49c8-c4fe-411b-89e5-e08bbcfea7db/volumes"
Sep 11 13:59:28.871945 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:28.871913    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:59:34.115604 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.115566    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc"]
Sep 11 13:59:34.116085 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.115955    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="056623be-e3cc-444e-afd7-c7238962c233" containerName="revoke-keys"
Sep 11 13:59:34.116085 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.115967    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="056623be-e3cc-444e-afd7-c7238962c233" containerName="revoke-keys"
Sep 11 13:59:34.116085 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.115980    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1da49c8-c4fe-411b-89e5-e08bbcfea7db" containerName="payload-processing"
Sep 11 13:59:34.116085 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.115987    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1da49c8-c4fe-411b-89e5-e08bbcfea7db" containerName="payload-processing"
Sep 11 13:59:34.116085 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.116044    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="056623be-e3cc-444e-afd7-c7238962c233" containerName="revoke-keys"
Sep 11 13:59:34.116085 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.116056    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1da49c8-c4fe-411b-89e5-e08bbcfea7db" containerName="payload-processing"
Sep 11 13:59:34.120071 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.120054    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc"
Sep 11 13:59:34.123865 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.123838    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:59:34.124190 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.124172    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:59:34.130040 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.130017    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc"]
Sep 11 13:59:34.188183 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.188142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3011df1e-bfe4-484a-b73b-61fdc0b9e59e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc\" (UID: \"3011df1e-bfe4-484a-b73b-61fdc0b9e59e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc"
Sep 11 13:59:34.289201 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.289165    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3011df1e-bfe4-484a-b73b-61fdc0b9e59e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc\" (UID: \"3011df1e-bfe4-484a-b73b-61fdc0b9e59e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc"
Sep 11 13:59:34.289809 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.289789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3011df1e-bfe4-484a-b73b-61fdc0b9e59e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc\" (UID: \"3011df1e-bfe4-484a-b73b-61fdc0b9e59e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc"
Sep 11 13:59:34.431175 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.431082    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc"
Sep 11 13:59:34.709298 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:34.709267    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc"]
Sep 11 13:59:34.710691 ip-10-0-142-132 kubenswrapper[2580]: W0911 13:59:34.710660    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3011df1e_bfe4_484a_b73b_61fdc0b9e59e.slice/crio-2e64e4f1c116fe4cf2d725e67ba1694512091d4906531b191eac062f1ecbddcd WatchSource:0}: Error finding container 2e64e4f1c116fe4cf2d725e67ba1694512091d4906531b191eac062f1ecbddcd: Status 404 returned error can't find the container with id 2e64e4f1c116fe4cf2d725e67ba1694512091d4906531b191eac062f1ecbddcd
Sep 11 13:59:35.382308 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:35.382264    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc" event={"ID":"3011df1e-bfe4-484a-b73b-61fdc0b9e59e","Type":"ContainerStarted","Data":"d3c48349ca9e74895d76a0f17322790995d90033f3eba03a3474178826ae82c1"}
Sep 11 13:59:35.382736 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:35.382381    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc" event={"ID":"3011df1e-bfe4-484a-b73b-61fdc0b9e59e","Type":"ContainerStarted","Data":"2e64e4f1c116fe4cf2d725e67ba1694512091d4906531b191eac062f1ecbddcd"}
Sep 11 13:59:35.402378 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:35.402325    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc" podStartSLOduration=1.402309141 podStartE2EDuration="1.402309141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:59:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:35.39985802 +0000 UTC m=+2294.618395512" watchObservedRunningTime="2026-09-11 13:59:35.402309141 +0000 UTC m=+2294.620846634"
Sep 11 13:59:36.389041 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:36.388998    2580 generic.go:356] "Generic (PLEG): container finished" podID="3011df1e-bfe4-484a-b73b-61fdc0b9e59e" containerID="d3c48349ca9e74895d76a0f17322790995d90033f3eba03a3474178826ae82c1" exitCode=0
Sep 11 13:59:36.389041 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:36.389044    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc" event={"ID":"3011df1e-bfe4-484a-b73b-61fdc0b9e59e","Type":"ContainerDied","Data":"d3c48349ca9e74895d76a0f17322790995d90033f3eba03a3474178826ae82c1"}
Sep 11 13:59:37.529868 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:37.529841    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc"
Sep 11 13:59:37.620634 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:37.620600    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3011df1e-bfe4-484a-b73b-61fdc0b9e59e-maas-api-service-ca\") pod \"3011df1e-bfe4-484a-b73b-61fdc0b9e59e\" (UID: \"3011df1e-bfe4-484a-b73b-61fdc0b9e59e\") "
Sep 11 13:59:37.620978 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:37.620954    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3011df1e-bfe4-484a-b73b-61fdc0b9e59e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3011df1e-bfe4-484a-b73b-61fdc0b9e59e" (UID: "3011df1e-bfe4-484a-b73b-61fdc0b9e59e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:59:37.722186 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:37.722098    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3011df1e-bfe4-484a-b73b-61fdc0b9e59e-maas-api-service-ca\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:59:38.401237 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:38.401200    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc" event={"ID":"3011df1e-bfe4-484a-b73b-61fdc0b9e59e","Type":"ContainerDied","Data":"2e64e4f1c116fe4cf2d725e67ba1694512091d4906531b191eac062f1ecbddcd"}
Sep 11 13:59:38.401237 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:38.401238    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc"
Sep 11 13:59:38.401449 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:38.401250    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e64e4f1c116fe4cf2d725e67ba1694512091d4906531b191eac062f1ecbddcd"
Sep 11 13:59:44.932957 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:44.932920    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"]
Sep 11 13:59:44.933539 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:44.933161    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp" podUID="014bbf30-59b1-45f2-a2dc-1449b1eafb1f" containerName="payload-processing" containerID="cri-o://9f987ed54d471ce6243129d8a1b766c5494c3b08a93aea795aebc12fee988a42" gracePeriod=30
Sep 11 13:59:45.023074 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.023038    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp" podUID="014bbf30-59b1-45f2-a2dc-1449b1eafb1f" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.81:9004: connect: connection refused"
Sep 11 13:59:45.123560 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.123526    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc"]
Sep 11 13:59:45.126623 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.126592    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-9edb53-r4tmc"]
Sep 11 13:59:45.188709 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.188647    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"
Sep 11 13:59:45.296010 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.295976    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6czst\" (UniqueName: \"kubernetes.io/projected/014bbf30-59b1-45f2-a2dc-1449b1eafb1f-kube-api-access-6czst\") pod \"014bbf30-59b1-45f2-a2dc-1449b1eafb1f\" (UID: \"014bbf30-59b1-45f2-a2dc-1449b1eafb1f\") "
Sep 11 13:59:45.296201 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.296094    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/014bbf30-59b1-45f2-a2dc-1449b1eafb1f-plugins-config-volume\") pod \"014bbf30-59b1-45f2-a2dc-1449b1eafb1f\" (UID: \"014bbf30-59b1-45f2-a2dc-1449b1eafb1f\") "
Sep 11 13:59:45.296481 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.296458    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/014bbf30-59b1-45f2-a2dc-1449b1eafb1f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "014bbf30-59b1-45f2-a2dc-1449b1eafb1f" (UID: "014bbf30-59b1-45f2-a2dc-1449b1eafb1f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:59:45.298088 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.298065    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/014bbf30-59b1-45f2-a2dc-1449b1eafb1f-kube-api-access-6czst" (OuterVolumeSpecName: "kube-api-access-6czst") pod "014bbf30-59b1-45f2-a2dc-1449b1eafb1f" (UID: "014bbf30-59b1-45f2-a2dc-1449b1eafb1f"). InnerVolumeSpecName "kube-api-access-6czst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:59:45.316196 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.316162    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3011df1e-bfe4-484a-b73b-61fdc0b9e59e" path="/var/lib/kubelet/pods/3011df1e-bfe4-484a-b73b-61fdc0b9e59e/volumes"
Sep 11 13:59:45.397743 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.397704    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/014bbf30-59b1-45f2-a2dc-1449b1eafb1f-plugins-config-volume\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:59:45.397743 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.397731    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6czst\" (UniqueName: \"kubernetes.io/projected/014bbf30-59b1-45f2-a2dc-1449b1eafb1f-kube-api-access-6czst\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 13:59:45.444845 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.444763    2580 generic.go:356] "Generic (PLEG): container finished" podID="014bbf30-59b1-45f2-a2dc-1449b1eafb1f" containerID="9f987ed54d471ce6243129d8a1b766c5494c3b08a93aea795aebc12fee988a42" exitCode=0
Sep 11 13:59:45.444845 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.444803    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"
Sep 11 13:59:45.445072 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.444840    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp" event={"ID":"014bbf30-59b1-45f2-a2dc-1449b1eafb1f","Type":"ContainerDied","Data":"9f987ed54d471ce6243129d8a1b766c5494c3b08a93aea795aebc12fee988a42"}
Sep 11 13:59:45.445072 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.444891    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp" event={"ID":"014bbf30-59b1-45f2-a2dc-1449b1eafb1f","Type":"ContainerDied","Data":"dcfa69f5182d8ea2770bfe0ff05d7a970740f3d98118ffabc4b9010ad8c4f973"}
Sep 11 13:59:45.445072 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.444931    2580 scope.go:117] "RemoveContainer" containerID="9f987ed54d471ce6243129d8a1b766c5494c3b08a93aea795aebc12fee988a42"
Sep 11 13:59:45.454147 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.454132    2580 scope.go:117] "RemoveContainer" containerID="9f987ed54d471ce6243129d8a1b766c5494c3b08a93aea795aebc12fee988a42"
Sep 11 13:59:45.454417 ip-10-0-142-132 kubenswrapper[2580]: E0911 13:59:45.454399    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f987ed54d471ce6243129d8a1b766c5494c3b08a93aea795aebc12fee988a42\": container with ID starting with 9f987ed54d471ce6243129d8a1b766c5494c3b08a93aea795aebc12fee988a42 not found: ID does not exist" containerID="9f987ed54d471ce6243129d8a1b766c5494c3b08a93aea795aebc12fee988a42"
Sep 11 13:59:45.454477 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.454428    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f987ed54d471ce6243129d8a1b766c5494c3b08a93aea795aebc12fee988a42"} err="failed to get container status \"9f987ed54d471ce6243129d8a1b766c5494c3b08a93aea795aebc12fee988a42\": rpc error: code = NotFound desc = could not find container \"9f987ed54d471ce6243129d8a1b766c5494c3b08a93aea795aebc12fee988a42\": container with ID starting with 9f987ed54d471ce6243129d8a1b766c5494c3b08a93aea795aebc12fee988a42 not found: ID does not exist"
Sep 11 13:59:45.463216 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.463193    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"]
Sep 11 13:59:45.469509 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:45.469487    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-9edb53-55cd9567b5-jvzmp"]
Sep 11 13:59:47.316325 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:47.316291    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="014bbf30-59b1-45f2-a2dc-1449b1eafb1f" path="/var/lib/kubelet/pods/014bbf30-59b1-45f2-a2dc-1449b1eafb1f/volumes"
Sep 11 13:59:47.658657 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:47.658554    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 13:59:57.655568 ip-10-0-142-132 kubenswrapper[2580]: I0911 13:59:57.655527    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:00:21.894546 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:00:21.894473    2580 scope.go:117] "RemoveContainer" containerID="b08f561e822684091ff7d5594453b5d1ecdad4e66686009dd137ab5023c8bc73"
Sep 11 14:01:17.802405 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:01:17.802367    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:01:21.511893 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:01:21.511867    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 14:01:21.524545 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:01:21.524520    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 14:01:44.548136 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:01:44.548098    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:01:52.351079 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:01:52.351044    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:01:59.362960 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:01:59.362920    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:02:07.376101 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:02:07.376063    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:02:21.761277 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:02:21.761236    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:02:28.655347 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:02:28.655310    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:03:56.582617 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:03:56.582579    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:04:03.254897 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:03.254859    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:04:12.062800 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.062767    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"]
Sep 11 14:04:12.063339 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.063198    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3011df1e-bfe4-484a-b73b-61fdc0b9e59e" containerName="revoke-keys"
Sep 11 14:04:12.063339 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.063215    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3011df1e-bfe4-484a-b73b-61fdc0b9e59e" containerName="revoke-keys"
Sep 11 14:04:12.063339 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.063226    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="014bbf30-59b1-45f2-a2dc-1449b1eafb1f" containerName="payload-processing"
Sep 11 14:04:12.063339 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.063232    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="014bbf30-59b1-45f2-a2dc-1449b1eafb1f" containerName="payload-processing"
Sep 11 14:04:12.063339 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.063288    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="014bbf30-59b1-45f2-a2dc-1449b1eafb1f" containerName="payload-processing"
Sep 11 14:04:12.063339 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.063297    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3011df1e-bfe4-484a-b73b-61fdc0b9e59e" containerName="revoke-keys"
Sep 11 14:04:12.066474 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.066451    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.069437 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.069414    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-1edd3094-kserve-self-signed-certs\""
Sep 11 14:04:12.080431 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.080396    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"]
Sep 11 14:04:12.170629 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.170595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-model-cache\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.170629 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.170635    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-home\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.170844 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.170663    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-tmp-dir\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.170844 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.170755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-dshm\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.170844 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.170800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8j8h\" (UniqueName: \"kubernetes.io/projected/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-kube-api-access-z8j8h\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.170844 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.170828    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-tls-certs\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.271636 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.271599    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-dshm\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.271864 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.271652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8j8h\" (UniqueName: \"kubernetes.io/projected/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-kube-api-access-z8j8h\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.271864 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.271767    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-tls-certs\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.271864 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.271815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-model-cache\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.271864 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.271853    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-home\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.272140 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.271937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-tmp-dir\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.272353 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.272319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-model-cache\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.272471 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.272337    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-home\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.272471 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.272364    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-tmp-dir\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.273851 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.273832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-dshm\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.274447 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.274425    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-tls-certs\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.282663 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.282636    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8j8h\" (UniqueName: \"kubernetes.io/projected/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-kube-api-access-z8j8h\") pod \"e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") " pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.381965 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.381859    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:12.467297 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.467259    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:04:12.655016 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.654989    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"]
Sep 11 14:04:12.657053 ip-10-0-142-132 kubenswrapper[2580]: W0911 14:04:12.657020    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b98f6cc_a8b6_48b0_bfb2_23031c3b364d.slice/crio-810f916a11bcb371fddb53c55180b58f066d31cacc8aed17ea86c30fe178417d WatchSource:0}: Error finding container 810f916a11bcb371fddb53c55180b58f066d31cacc8aed17ea86c30fe178417d: Status 404 returned error can't find the container with id 810f916a11bcb371fddb53c55180b58f066d31cacc8aed17ea86c30fe178417d
Sep 11 14:04:12.912335 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.912234    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr" event={"ID":"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d","Type":"ContainerStarted","Data":"05df5b17a5d31f4e605bbe8f287d0d14752ef98f46682f8dbd5092cee8a6aaa4"}
Sep 11 14:04:12.912335 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.912284    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr" event={"ID":"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d","Type":"ContainerStarted","Data":"810f916a11bcb371fddb53c55180b58f066d31cacc8aed17ea86c30fe178417d"}
Sep 11 14:04:12.941828 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:12.941776    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr" podStartSLOduration=0.941756487 podStartE2EDuration="941.756487ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:04:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:04:12.938774407 +0000 UTC m=+2572.157311898" watchObservedRunningTime="2026-09-11 14:04:12.941756487 +0000 UTC m=+2572.160293980"
Sep 11 14:04:19.551012 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:19.550980    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:04:20.532152 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.532114    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"]
Sep 11 14:04:20.535586 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.535563    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.538356 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.538331    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-1edd3094-kserve-self-signed-certs\""
Sep 11 14:04:20.546797 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.546771    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"]
Sep 11 14:04:20.651777 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.651744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-dshm\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.651777 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.651786    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-tls-certs\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.652224 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.651821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-model-cache\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.652224 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.651837    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-home\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.652224 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.651859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-tmp-dir\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.652224 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.651875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vjdh\" (UniqueName: \"kubernetes.io/projected/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-kube-api-access-2vjdh\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.753346 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.753305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-model-cache\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.753530 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.753364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-home\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.753530 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.753408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-tmp-dir\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.753530 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.753432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vjdh\" (UniqueName: \"kubernetes.io/projected/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-kube-api-access-2vjdh\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.753710 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.753530    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-dshm\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.753710 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.753574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-tls-certs\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.753710 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.753648    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-model-cache\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.753845 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.753713    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-home\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.753845 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.753761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-tmp-dir\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.755848 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.755815    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-dshm\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.756075 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.756056    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-tls-certs\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.762732 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.762711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vjdh\" (UniqueName: \"kubernetes.io/projected/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-kube-api-access-2vjdh\") pod \"e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") " pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:20.845670 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:20.845637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:21.120323 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:21.120296    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"]
Sep 11 14:04:21.121919 ip-10-0-142-132 kubenswrapper[2580]: W0911 14:04:21.121874    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67427ee6_2cd2_431b_b6e9_8edfe0f65b43.slice/crio-68d2d341078743ab43111a8bb3da8fb899bd581da20c968aef3968a9dab52ec8 WatchSource:0}: Error finding container 68d2d341078743ab43111a8bb3da8fb899bd581da20c968aef3968a9dab52ec8: Status 404 returned error can't find the container with id 68d2d341078743ab43111a8bb3da8fb899bd581da20c968aef3968a9dab52ec8
Sep 11 14:04:21.967117 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:21.967074    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j" event={"ID":"67427ee6-2cd2-431b-b6e9-8edfe0f65b43","Type":"ContainerStarted","Data":"afb2f9b457bf048a58c8121f7751850b63315c3077b0657b6952cdb9547bd475"}
Sep 11 14:04:21.967503 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:21.967126    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j" event={"ID":"67427ee6-2cd2-431b-b6e9-8edfe0f65b43","Type":"ContainerStarted","Data":"68d2d341078743ab43111a8bb3da8fb899bd581da20c968aef3968a9dab52ec8"}
Sep 11 14:04:21.997781 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:21.997735    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j" podStartSLOduration=1.997719403 podStartE2EDuration="1.997719403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:04:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:04:21.995764008 +0000 UTC m=+2581.214301501" watchObservedRunningTime="2026-09-11 14:04:21.997719403 +0000 UTC m=+2581.216256959"
Sep 11 14:04:22.382710 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:22.382679    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:22.382887 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:22.382726    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:22.395446 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:22.395422    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:22.984059 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:22.984021    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:28.232188 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:28.232151    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:04:30.846011 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:30.845968    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:30.846011 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:30.846024    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:30.858724 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:30.858696    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:31.028454 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:31.028422    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:42.645306 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:42.645264    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"]
Sep 11 14:04:44.038173 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:44.038140    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"]
Sep 11 14:04:45.847827 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:45.847787    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:04:53.667362 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:53.667328    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:04:57.664331 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:57.664256    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr" podUID="2b98f6cc-a8b6-48b0-bfb2-23031c3b364d" containerName="main" containerID="cri-o://05df5b17a5d31f4e605bbe8f287d0d14752ef98f46682f8dbd5092cee8a6aaa4" gracePeriod=45
Sep 11 14:04:57.917730 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:57.917669    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:58.000051 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.000020    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8j8h\" (UniqueName: \"kubernetes.io/projected/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-kube-api-access-z8j8h\") pod \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") "
Sep 11 14:04:58.000051 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.000054    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-model-cache\") pod \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") "
Sep 11 14:04:58.000313 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.000091    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-home\") pod \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") "
Sep 11 14:04:58.000313 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.000106    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-dshm\") pod \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") "
Sep 11 14:04:58.000313 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.000127    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-tls-certs\") pod \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") "
Sep 11 14:04:58.000313 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.000187    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-tmp-dir\") pod \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\" (UID: \"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d\") "
Sep 11 14:04:58.000534 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.000359    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-home" (OuterVolumeSpecName: "home") pod "2b98f6cc-a8b6-48b0-bfb2-23031c3b364d" (UID: "2b98f6cc-a8b6-48b0-bfb2-23031c3b364d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:58.000534 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.000377    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-model-cache" (OuterVolumeSpecName: "model-cache") pod "2b98f6cc-a8b6-48b0-bfb2-23031c3b364d" (UID: "2b98f6cc-a8b6-48b0-bfb2-23031c3b364d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:58.000534 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.000501    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2b98f6cc-a8b6-48b0-bfb2-23031c3b364d" (UID: "2b98f6cc-a8b6-48b0-bfb2-23031c3b364d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:58.000534 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.000525    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 14:04:58.000683 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.000538    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 14:04:58.002425 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.002395    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-dshm" (OuterVolumeSpecName: "dshm") pod "2b98f6cc-a8b6-48b0-bfb2-23031c3b364d" (UID: "2b98f6cc-a8b6-48b0-bfb2-23031c3b364d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:58.002522 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.002442    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-kube-api-access-z8j8h" (OuterVolumeSpecName: "kube-api-access-z8j8h") pod "2b98f6cc-a8b6-48b0-bfb2-23031c3b364d" (UID: "2b98f6cc-a8b6-48b0-bfb2-23031c3b364d"). InnerVolumeSpecName "kube-api-access-z8j8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:04:58.002522 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.002463    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2b98f6cc-a8b6-48b0-bfb2-23031c3b364d" (UID: "2b98f6cc-a8b6-48b0-bfb2-23031c3b364d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:04:58.101425 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.101372    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8j8h\" (UniqueName: \"kubernetes.io/projected/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-kube-api-access-z8j8h\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 14:04:58.101425 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.101417    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 14:04:58.101425 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.101427    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 14:04:58.101425 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.101436    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 14:04:58.163672 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.163640    2580 generic.go:356] "Generic (PLEG): container finished" podID="2b98f6cc-a8b6-48b0-bfb2-23031c3b364d" containerID="05df5b17a5d31f4e605bbe8f287d0d14752ef98f46682f8dbd5092cee8a6aaa4" exitCode=0
Sep 11 14:04:58.163865 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.163686    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"
Sep 11 14:04:58.163865 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.163728    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr" event={"ID":"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d","Type":"ContainerDied","Data":"05df5b17a5d31f4e605bbe8f287d0d14752ef98f46682f8dbd5092cee8a6aaa4"}
Sep 11 14:04:58.163865 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.163771    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr" event={"ID":"2b98f6cc-a8b6-48b0-bfb2-23031c3b364d","Type":"ContainerDied","Data":"810f916a11bcb371fddb53c55180b58f066d31cacc8aed17ea86c30fe178417d"}
Sep 11 14:04:58.163865 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.163794    2580 scope.go:117] "RemoveContainer" containerID="05df5b17a5d31f4e605bbe8f287d0d14752ef98f46682f8dbd5092cee8a6aaa4"
Sep 11 14:04:58.173103 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.173086    2580 scope.go:117] "RemoveContainer" containerID="05df5b17a5d31f4e605bbe8f287d0d14752ef98f46682f8dbd5092cee8a6aaa4"
Sep 11 14:04:58.173338 ip-10-0-142-132 kubenswrapper[2580]: E0911 14:04:58.173321    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05df5b17a5d31f4e605bbe8f287d0d14752ef98f46682f8dbd5092cee8a6aaa4\": container with ID starting with 05df5b17a5d31f4e605bbe8f287d0d14752ef98f46682f8dbd5092cee8a6aaa4 not found: ID does not exist" containerID="05df5b17a5d31f4e605bbe8f287d0d14752ef98f46682f8dbd5092cee8a6aaa4"
Sep 11 14:04:58.173470 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.173348    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05df5b17a5d31f4e605bbe8f287d0d14752ef98f46682f8dbd5092cee8a6aaa4"} err="failed to get container status \"05df5b17a5d31f4e605bbe8f287d0d14752ef98f46682f8dbd5092cee8a6aaa4\": rpc error: code = NotFound desc = could not find container \"05df5b17a5d31f4e605bbe8f287d0d14752ef98f46682f8dbd5092cee8a6aaa4\": container with ID starting with 05df5b17a5d31f4e605bbe8f287d0d14752ef98f46682f8dbd5092cee8a6aaa4 not found: ID does not exist"
Sep 11 14:04:58.191056 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.191028    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"]
Sep 11 14:04:58.196874 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:58.196852    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-1edd3094-kserve-b6d8f6955-2wnbr"]
Sep 11 14:04:59.057186 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.057135    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j" podUID="67427ee6-2cd2-431b-b6e9-8edfe0f65b43" containerName="main" containerID="cri-o://afb2f9b457bf048a58c8121f7751850b63315c3077b0657b6952cdb9547bd475" gracePeriod=45
Sep 11 14:04:59.307535 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.307479    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:04:59.315213 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.315185    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b98f6cc-a8b6-48b0-bfb2-23031c3b364d" path="/var/lib/kubelet/pods/2b98f6cc-a8b6-48b0-bfb2-23031c3b364d/volumes"
Sep 11 14:04:59.411242 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.411208    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-tls-certs\") pod \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") "
Sep 11 14:04:59.411242 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.411242    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-home\") pod \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") "
Sep 11 14:04:59.411502 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.411269    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-dshm\") pod \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") "
Sep 11 14:04:59.411502 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.411301    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vjdh\" (UniqueName: \"kubernetes.io/projected/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-kube-api-access-2vjdh\") pod \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") "
Sep 11 14:04:59.411502 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.411338    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-tmp-dir\") pod \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") "
Sep 11 14:04:59.411502 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.411357    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-model-cache\") pod \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\" (UID: \"67427ee6-2cd2-431b-b6e9-8edfe0f65b43\") "
Sep 11 14:04:59.411726 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.411500    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-home" (OuterVolumeSpecName: "home") pod "67427ee6-2cd2-431b-b6e9-8edfe0f65b43" (UID: "67427ee6-2cd2-431b-b6e9-8edfe0f65b43"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:59.411726 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.411627    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "67427ee6-2cd2-431b-b6e9-8edfe0f65b43" (UID: "67427ee6-2cd2-431b-b6e9-8edfe0f65b43"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:59.411726 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.411666    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-model-cache" (OuterVolumeSpecName: "model-cache") pod "67427ee6-2cd2-431b-b6e9-8edfe0f65b43" (UID: "67427ee6-2cd2-431b-b6e9-8edfe0f65b43"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:59.411726 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.411720    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-home\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 14:04:59.411885 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.411734    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-tmp-dir\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 14:04:59.413489 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.413456    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-dshm" (OuterVolumeSpecName: "dshm") pod "67427ee6-2cd2-431b-b6e9-8edfe0f65b43" (UID: "67427ee6-2cd2-431b-b6e9-8edfe0f65b43"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:04:59.413595 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.413531    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-kube-api-access-2vjdh" (OuterVolumeSpecName: "kube-api-access-2vjdh") pod "67427ee6-2cd2-431b-b6e9-8edfe0f65b43" (UID: "67427ee6-2cd2-431b-b6e9-8edfe0f65b43"). InnerVolumeSpecName "kube-api-access-2vjdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:04:59.413595 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.413557    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "67427ee6-2cd2-431b-b6e9-8edfe0f65b43" (UID: "67427ee6-2cd2-431b-b6e9-8edfe0f65b43"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:04:59.512529 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.512491    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-dshm\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 14:04:59.512529 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.512524    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vjdh\" (UniqueName: \"kubernetes.io/projected/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-kube-api-access-2vjdh\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 14:04:59.512529 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.512533    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-model-cache\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 14:04:59.512755 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:04:59.512544    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67427ee6-2cd2-431b-b6e9-8edfe0f65b43-tls-certs\") on node \"ip-10-0-142-132.ec2.internal\" DevicePath \"\""
Sep 11 14:05:00.176390 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:00.176351    2580 generic.go:356] "Generic (PLEG): container finished" podID="67427ee6-2cd2-431b-b6e9-8edfe0f65b43" containerID="afb2f9b457bf048a58c8121f7751850b63315c3077b0657b6952cdb9547bd475" exitCode=0
Sep 11 14:05:00.176390 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:00.176397    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"
Sep 11 14:05:00.176947 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:00.176447    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j" event={"ID":"67427ee6-2cd2-431b-b6e9-8edfe0f65b43","Type":"ContainerDied","Data":"afb2f9b457bf048a58c8121f7751850b63315c3077b0657b6952cdb9547bd475"}
Sep 11 14:05:00.176947 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:00.176502    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j" event={"ID":"67427ee6-2cd2-431b-b6e9-8edfe0f65b43","Type":"ContainerDied","Data":"68d2d341078743ab43111a8bb3da8fb899bd581da20c968aef3968a9dab52ec8"}
Sep 11 14:05:00.176947 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:00.176527    2580 scope.go:117] "RemoveContainer" containerID="afb2f9b457bf048a58c8121f7751850b63315c3077b0657b6952cdb9547bd475"
Sep 11 14:05:00.186388 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:00.186369    2580 scope.go:117] "RemoveContainer" containerID="afb2f9b457bf048a58c8121f7751850b63315c3077b0657b6952cdb9547bd475"
Sep 11 14:05:00.186705 ip-10-0-142-132 kubenswrapper[2580]: E0911 14:05:00.186684    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afb2f9b457bf048a58c8121f7751850b63315c3077b0657b6952cdb9547bd475\": container with ID starting with afb2f9b457bf048a58c8121f7751850b63315c3077b0657b6952cdb9547bd475 not found: ID does not exist" containerID="afb2f9b457bf048a58c8121f7751850b63315c3077b0657b6952cdb9547bd475"
Sep 11 14:05:00.186789 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:00.186714    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afb2f9b457bf048a58c8121f7751850b63315c3077b0657b6952cdb9547bd475"} err="failed to get container status \"afb2f9b457bf048a58c8121f7751850b63315c3077b0657b6952cdb9547bd475\": rpc error: code = NotFound desc = could not find container \"afb2f9b457bf048a58c8121f7751850b63315c3077b0657b6952cdb9547bd475\": container with ID starting with afb2f9b457bf048a58c8121f7751850b63315c3077b0657b6952cdb9547bd475 not found: ID does not exist"
Sep 11 14:05:00.207164 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:00.207136    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"]
Sep 11 14:05:00.216949 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:00.216926    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-1edd3094-kserve-6499f4864c-d2q4j"]
Sep 11 14:05:01.315390 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:01.315359    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67427ee6-2cd2-431b-b6e9-8edfe0f65b43" path="/var/lib/kubelet/pods/67427ee6-2cd2-431b-b6e9-8edfe0f65b43/volumes"
Sep 11 14:05:01.658540 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:01.658452    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:05:10.391939 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.391891    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"]
Sep 11 14:05:10.392325 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.392253    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b98f6cc-a8b6-48b0-bfb2-23031c3b364d" containerName="main"
Sep 11 14:05:10.392325 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.392264    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b98f6cc-a8b6-48b0-bfb2-23031c3b364d" containerName="main"
Sep 11 14:05:10.392325 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.392275    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67427ee6-2cd2-431b-b6e9-8edfe0f65b43" containerName="main"
Sep 11 14:05:10.392325 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.392281    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="67427ee6-2cd2-431b-b6e9-8edfe0f65b43" containerName="main"
Sep 11 14:05:10.392462 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.392343    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="67427ee6-2cd2-431b-b6e9-8edfe0f65b43" containerName="main"
Sep 11 14:05:10.392462 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.392353    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b98f6cc-a8b6-48b0-bfb2-23031c3b364d" containerName="main"
Sep 11 14:05:10.396447 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.396429    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:10.399281 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.399260    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 14:05:10.400356 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.400339    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6zc5p\""
Sep 11 14:05:10.400424 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.400344    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 14:05:10.400474 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.400422    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 14:05:10.400474 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.400461    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 14:05:10.421371 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.421347    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"]
Sep 11 14:05:10.487865 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.487835    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:05:10.519629 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.519591    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncfnk\" (UniqueName: \"kubernetes.io/projected/42939f2e-de8c-48df-bc08-8b1e7e5bc824-kube-api-access-ncfnk\") pod \"maas-api-f8bb4c8cc-nt4h5\" (UID: \"42939f2e-de8c-48df-bc08-8b1e7e5bc824\") " pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:10.519787 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.519671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42939f2e-de8c-48df-bc08-8b1e7e5bc824-openshift-service-ca\") pod \"maas-api-f8bb4c8cc-nt4h5\" (UID: \"42939f2e-de8c-48df-bc08-8b1e7e5bc824\") " pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:10.519787 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.519698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42939f2e-de8c-48df-bc08-8b1e7e5bc824-maas-api-tls\") pod \"maas-api-f8bb4c8cc-nt4h5\" (UID: \"42939f2e-de8c-48df-bc08-8b1e7e5bc824\") " pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:10.519787 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.519720    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42939f2e-de8c-48df-bc08-8b1e7e5bc824-metrics-certs\") pod \"maas-api-f8bb4c8cc-nt4h5\" (UID: \"42939f2e-de8c-48df-bc08-8b1e7e5bc824\") " pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:10.620985 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.620945    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42939f2e-de8c-48df-bc08-8b1e7e5bc824-openshift-service-ca\") pod \"maas-api-f8bb4c8cc-nt4h5\" (UID: \"42939f2e-de8c-48df-bc08-8b1e7e5bc824\") " pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:10.621155 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.620993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42939f2e-de8c-48df-bc08-8b1e7e5bc824-maas-api-tls\") pod \"maas-api-f8bb4c8cc-nt4h5\" (UID: \"42939f2e-de8c-48df-bc08-8b1e7e5bc824\") " pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:10.621155 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.621021    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42939f2e-de8c-48df-bc08-8b1e7e5bc824-metrics-certs\") pod \"maas-api-f8bb4c8cc-nt4h5\" (UID: \"42939f2e-de8c-48df-bc08-8b1e7e5bc824\") " pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:10.621155 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.621108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncfnk\" (UniqueName: \"kubernetes.io/projected/42939f2e-de8c-48df-bc08-8b1e7e5bc824-kube-api-access-ncfnk\") pod \"maas-api-f8bb4c8cc-nt4h5\" (UID: \"42939f2e-de8c-48df-bc08-8b1e7e5bc824\") " pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:10.621729 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.621698    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42939f2e-de8c-48df-bc08-8b1e7e5bc824-openshift-service-ca\") pod \"maas-api-f8bb4c8cc-nt4h5\" (UID: \"42939f2e-de8c-48df-bc08-8b1e7e5bc824\") " pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:10.623585 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.623560    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42939f2e-de8c-48df-bc08-8b1e7e5bc824-maas-api-tls\") pod \"maas-api-f8bb4c8cc-nt4h5\" (UID: \"42939f2e-de8c-48df-bc08-8b1e7e5bc824\") " pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:10.623691 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.623626    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42939f2e-de8c-48df-bc08-8b1e7e5bc824-metrics-certs\") pod \"maas-api-f8bb4c8cc-nt4h5\" (UID: \"42939f2e-de8c-48df-bc08-8b1e7e5bc824\") " pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:10.631860 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.631835    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncfnk\" (UniqueName: \"kubernetes.io/projected/42939f2e-de8c-48df-bc08-8b1e7e5bc824-kube-api-access-ncfnk\") pod \"maas-api-f8bb4c8cc-nt4h5\" (UID: \"42939f2e-de8c-48df-bc08-8b1e7e5bc824\") " pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:10.706775 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.706680    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:10.989522 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.989494    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"]
Sep 11 14:05:10.991027 ip-10-0-142-132 kubenswrapper[2580]: W0911 14:05:10.991000    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42939f2e_de8c_48df_bc08_8b1e7e5bc824.slice/crio-4fcec8dc473f7f5535890ef2050d8035f08fed4140c250aa2e92d6a749c15635 WatchSource:0}: Error finding container 4fcec8dc473f7f5535890ef2050d8035f08fed4140c250aa2e92d6a749c15635: Status 404 returned error can't find the container with id 4fcec8dc473f7f5535890ef2050d8035f08fed4140c250aa2e92d6a749c15635
Sep 11 14:05:10.992436 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:10.992413    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:05:11.237713 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:11.237678    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5" event={"ID":"42939f2e-de8c-48df-bc08-8b1e7e5bc824","Type":"ContainerStarted","Data":"4fcec8dc473f7f5535890ef2050d8035f08fed4140c250aa2e92d6a749c15635"}
Sep 11 14:05:14.258305 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:14.258267    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5" event={"ID":"42939f2e-de8c-48df-bc08-8b1e7e5bc824","Type":"ContainerStarted","Data":"ea9902ad8c67d1db4f0b20cafada36a042fa81870ed89deb2c5bd2253c35fc36"}
Sep 11 14:05:14.258671 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:14.258426    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:14.277404 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:14.277354    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5" podStartSLOduration=1.9788041010000001 podStartE2EDuration="4.277338689s" totalImagesPullingTime="2.298534588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:05:14.276185495 +0000 UTC m=+2633.494723000" watchObservedRunningTime="2026-09-11 14:05:14.277338689 +0000 UTC m=+2633.495876181"
Sep 11 14:05:17.464868 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:17.464830    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:05:20.270277 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:20.270246    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-f8bb4c8cc-nt4h5"
Sep 11 14:05:21.956442 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:21.956411    2580 scope.go:117] "RemoveContainer" containerID="1a24e1e65d6eccb03c3c57fba488d22839f4323bc991f985d47ece8dfd48285e"
Sep 11 14:05:21.965103 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:21.965079    2580 scope.go:117] "RemoveContainer" containerID="f860f8503d65d422bd30d990db639b2255d2935a672ad6c371d8332f53cc86ec"
Sep 11 14:05:23.446250 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:23.446212    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:05:30.472595 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:30.472556    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-kbbvr"]
Sep 11 14:05:33.553269 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:33.553231    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-698d8fd7fb-r6z5s_782d2109-57c6-4ec6-8ea4-3e6d26c19e1d/authorino/0.log"
Sep 11 14:05:40.510493 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:40.510460    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9576ccb85-fkn2f_4b978d74-2efb-4262-b0a0-ad66cc441e03/copy-manifests/0.log"
Sep 11 14:05:40.518719 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:40.518693    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9576ccb85-fkn2f_4b978d74-2efb-4262-b0a0-ad66cc441e03/manager/0.log"
Sep 11 14:05:41.213644 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:41.213617    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-f8bb4c8cc-nt4h5_42939f2e-de8c-48df-bc08-8b1e7e5bc824/maas-api/0.log"
Sep 11 14:05:41.510660 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:41.510586    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29818905-q2bv2_e5fbe1c1-68f4-436b-bf35-47b7694d82b6/cleanup/0.log"
Sep 11 14:05:42.711738 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:42.711703    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-698d8fd7fb-r6z5s_782d2109-57c6-4ec6-8ea4-3e6d26c19e1d/authorino/0.log"
Sep 11 14:05:43.103546 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:43.103521    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-qd7cg_750c0e85-0552-4e45-82aa-ee81c9b6215e/kuadrant-console-plugin/0.log"
Sep 11 14:05:43.387067 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:43.386995    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-kbbvr_3d4fcc93-924f-439a-9eec-967c670a886e/limitador/0.log"
Sep 11 14:05:44.325617 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:44.325585    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c9bdb7954-t6r2f_7e7c7651-a31f-413e-99d5-3e61b09e46d5/kube-auth-proxy/0.log"
Sep 11 14:05:45.790056 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:45.790025    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-fkcct_5b185795-5415-45d2-84e3-acef37f5dfe9/main/0.log"
Sep 11 14:05:46.192467 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:46.192438    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-wmzb9_0df0572b-fff0-4b71-ac1e-bf358a8f43e4/main/0.log"
Sep 11 14:05:53.020275 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:53.020240    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jwnld/must-gather-t889m"]
Sep 11 14:05:53.027414 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:53.027390    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jwnld/must-gather-t889m"
Sep 11 14:05:53.034842 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:53.034665    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jwnld\"/\"openshift-service-ca.crt\""
Sep 11 14:05:53.034842 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:53.034722    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jwnld\"/\"default-dockercfg-zb6z7\""
Sep 11 14:05:53.034842 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:53.034982    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jwnld\"/\"kube-root-ca.crt\""
Sep 11 14:05:53.036638 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:53.036618    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jwnld/must-gather-t889m"]
Sep 11 14:05:53.087072 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:53.087038    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7e21de9e-d40c-41ab-9255-f36376745e2d-must-gather-output\") pod \"must-gather-t889m\" (UID: \"7e21de9e-d40c-41ab-9255-f36376745e2d\") " pod="openshift-must-gather-jwnld/must-gather-t889m"
Sep 11 14:05:53.087288 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:53.087076    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zvdc\" (UniqueName: \"kubernetes.io/projected/7e21de9e-d40c-41ab-9255-f36376745e2d-kube-api-access-2zvdc\") pod \"must-gather-t889m\" (UID: \"7e21de9e-d40c-41ab-9255-f36376745e2d\") " pod="openshift-must-gather-jwnld/must-gather-t889m"
Sep 11 14:05:53.188591 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:53.188559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zvdc\" (UniqueName: \"kubernetes.io/projected/7e21de9e-d40c-41ab-9255-f36376745e2d-kube-api-access-2zvdc\") pod \"must-gather-t889m\" (UID: \"7e21de9e-d40c-41ab-9255-f36376745e2d\") " pod="openshift-must-gather-jwnld/must-gather-t889m"
Sep 11 14:05:53.188776 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:53.188689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7e21de9e-d40c-41ab-9255-f36376745e2d-must-gather-output\") pod \"must-gather-t889m\" (UID: \"7e21de9e-d40c-41ab-9255-f36376745e2d\") " pod="openshift-must-gather-jwnld/must-gather-t889m"
Sep 11 14:05:53.189043 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:53.189022    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7e21de9e-d40c-41ab-9255-f36376745e2d-must-gather-output\") pod \"must-gather-t889m\" (UID: \"7e21de9e-d40c-41ab-9255-f36376745e2d\") " pod="openshift-must-gather-jwnld/must-gather-t889m"
Sep 11 14:05:53.200587 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:53.200562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zvdc\" (UniqueName: \"kubernetes.io/projected/7e21de9e-d40c-41ab-9255-f36376745e2d-kube-api-access-2zvdc\") pod \"must-gather-t889m\" (UID: \"7e21de9e-d40c-41ab-9255-f36376745e2d\") " pod="openshift-must-gather-jwnld/must-gather-t889m"
Sep 11 14:05:53.337436 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:53.337405    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jwnld/must-gather-t889m"
Sep 11 14:05:53.613259 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:53.613231    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jwnld/must-gather-t889m"]
Sep 11 14:05:53.614563 ip-10-0-142-132 kubenswrapper[2580]: W0911 14:05:53.614534    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e21de9e_d40c_41ab_9255_f36376745e2d.slice/crio-22f82e5355e760c98d8689897621e0223a5775d501dca112801fe2b46f9cc708 WatchSource:0}: Error finding container 22f82e5355e760c98d8689897621e0223a5775d501dca112801fe2b46f9cc708: Status 404 returned error can't find the container with id 22f82e5355e760c98d8689897621e0223a5775d501dca112801fe2b46f9cc708
Sep 11 14:05:54.474776 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:54.474735    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jwnld/must-gather-t889m" event={"ID":"7e21de9e-d40c-41ab-9255-f36376745e2d","Type":"ContainerStarted","Data":"22f82e5355e760c98d8689897621e0223a5775d501dca112801fe2b46f9cc708"}
Sep 11 14:05:55.482768 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:55.482728    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jwnld/must-gather-t889m" event={"ID":"7e21de9e-d40c-41ab-9255-f36376745e2d","Type":"ContainerStarted","Data":"b8cdcf7d776f014b00f90909f11da8e17f9f71bae6551a21693da36539307cec"}
Sep 11 14:05:55.483226 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:55.482783    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jwnld/must-gather-t889m" event={"ID":"7e21de9e-d40c-41ab-9255-f36376745e2d","Type":"ContainerStarted","Data":"a0008a10d03cf2b1edca9ac64ddc032fc3ec9b02c28ed23ce165fee66077fe93"}
Sep 11 14:05:55.505474 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:55.505418    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jwnld/must-gather-t889m" podStartSLOduration=2.563217103 podStartE2EDuration="3.505401132s" totalImagesPullingTime="942.184029ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:05:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:05:55.502737897 +0000 UTC m=+2674.721275390" watchObservedRunningTime="2026-09-11 14:05:55.505401132 +0000 UTC m=+2674.723938625"
Sep 11 14:05:56.317867 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:56.317836    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fld56_aa614955-d70e-4ab5-b9fb-6a58f218f477/global-pull-secret-syncer/0.log"
Sep 11 14:05:56.523894 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:56.523861    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xbv6v_000a37ec-f62c-4808-a1ec-2992d0f919a4/konnectivity-agent/0.log"
Sep 11 14:05:56.624828 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:05:56.624740    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-132.ec2.internal_102eeadc7fecd8005cb07392235d7fc0/haproxy/0.log"
Sep 11 14:06:01.374927 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:01.374382    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-698d8fd7fb-r6z5s_782d2109-57c6-4ec6-8ea4-3e6d26c19e1d/authorino/0.log"
Sep 11 14:06:01.473982 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:01.473949    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-qd7cg_750c0e85-0552-4e45-82aa-ee81c9b6215e/kuadrant-console-plugin/0.log"
Sep 11 14:06:01.552489 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:01.552457    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-kbbvr_3d4fcc93-924f-439a-9eec-967c670a886e/limitador/0.log"
Sep 11 14:06:03.140425 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:03.140396    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-5wm6g_b76062ff-6ab0-4d3a-b139-12b777cb0ce6/cluster-monitoring-operator/0.log"
Sep 11 14:06:03.304654 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:03.304619    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-zkgsj_4ca70175-727d-4083-859c-095e0d48e811/monitoring-plugin/0.log"
Sep 11 14:06:03.529268 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:03.529194    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q2lnk_44626f2f-a336-45c7-8623-31f5f7ac93ca/node-exporter/0.log"
Sep 11 14:06:03.559469 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:03.559442    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q2lnk_44626f2f-a336-45c7-8623-31f5f7ac93ca/kube-rbac-proxy/0.log"
Sep 11 14:06:03.582449 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:03.582414    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q2lnk_44626f2f-a336-45c7-8623-31f5f7ac93ca/init-textfile/0.log"
Sep 11 14:06:03.968272 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:03.968246    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cc665f996-jzwh2_43e0b77a-f94e-4b10-b701-81445648a729/telemeter-client/0.log"
Sep 11 14:06:03.992691 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:03.992664    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cc665f996-jzwh2_43e0b77a-f94e-4b10-b701-81445648a729/reload/0.log"
Sep 11 14:06:04.016049 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:04.016017    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cc665f996-jzwh2_43e0b77a-f94e-4b10-b701-81445648a729/kube-rbac-proxy/0.log"
Sep 11 14:06:05.537246 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:05.537216    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-tn4r8_11bc16f7-1763-49ec-ae74-2783e7970924/networking-console-plugin/0.log"
Sep 11 14:06:06.310206 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.310165    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"]
Sep 11 14:06:06.316235 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.316206    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.336506 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.336473    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"]
Sep 11 14:06:06.353748 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.353720    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 14:06:06.358949 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.358922    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/3.log"
Sep 11 14:06:06.429419 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.429376    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18fbad89-9c36-49a5-9c01-616d4fec7fc0-podres\") pod \"perf-node-gather-daemonset-5q7r5\" (UID: \"18fbad89-9c36-49a5-9c01-616d4fec7fc0\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.429767 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.429745    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18fbad89-9c36-49a5-9c01-616d4fec7fc0-sys\") pod \"perf-node-gather-daemonset-5q7r5\" (UID: \"18fbad89-9c36-49a5-9c01-616d4fec7fc0\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.430391 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.430371    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18fbad89-9c36-49a5-9c01-616d4fec7fc0-lib-modules\") pod \"perf-node-gather-daemonset-5q7r5\" (UID: \"18fbad89-9c36-49a5-9c01-616d4fec7fc0\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.430516 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.430484    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18fbad89-9c36-49a5-9c01-616d4fec7fc0-proc\") pod \"perf-node-gather-daemonset-5q7r5\" (UID: \"18fbad89-9c36-49a5-9c01-616d4fec7fc0\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.430699 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.430674    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6t6p\" (UniqueName: \"kubernetes.io/projected/18fbad89-9c36-49a5-9c01-616d4fec7fc0-kube-api-access-h6t6p\") pod \"perf-node-gather-daemonset-5q7r5\" (UID: \"18fbad89-9c36-49a5-9c01-616d4fec7fc0\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.531570 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.531523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18fbad89-9c36-49a5-9c01-616d4fec7fc0-podres\") pod \"perf-node-gather-daemonset-5q7r5\" (UID: \"18fbad89-9c36-49a5-9c01-616d4fec7fc0\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.531745 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.531590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18fbad89-9c36-49a5-9c01-616d4fec7fc0-sys\") pod \"perf-node-gather-daemonset-5q7r5\" (UID: \"18fbad89-9c36-49a5-9c01-616d4fec7fc0\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.531745 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.531643    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18fbad89-9c36-49a5-9c01-616d4fec7fc0-lib-modules\") pod \"perf-node-gather-daemonset-5q7r5\" (UID: \"18fbad89-9c36-49a5-9c01-616d4fec7fc0\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.531745 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.531678    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18fbad89-9c36-49a5-9c01-616d4fec7fc0-proc\") pod \"perf-node-gather-daemonset-5q7r5\" (UID: \"18fbad89-9c36-49a5-9c01-616d4fec7fc0\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.531745 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.531699    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18fbad89-9c36-49a5-9c01-616d4fec7fc0-sys\") pod \"perf-node-gather-daemonset-5q7r5\" (UID: \"18fbad89-9c36-49a5-9c01-616d4fec7fc0\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.531954 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.531757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6t6p\" (UniqueName: \"kubernetes.io/projected/18fbad89-9c36-49a5-9c01-616d4fec7fc0-kube-api-access-h6t6p\") pod \"perf-node-gather-daemonset-5q7r5\" (UID: \"18fbad89-9c36-49a5-9c01-616d4fec7fc0\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.531954 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.531702    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/18fbad89-9c36-49a5-9c01-616d4fec7fc0-podres\") pod \"perf-node-gather-daemonset-5q7r5\" (UID: \"18fbad89-9c36-49a5-9c01-616d4fec7fc0\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.531954 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.531771    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/18fbad89-9c36-49a5-9c01-616d4fec7fc0-proc\") pod \"perf-node-gather-daemonset-5q7r5\" (UID: \"18fbad89-9c36-49a5-9c01-616d4fec7fc0\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.531954 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.531812    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18fbad89-9c36-49a5-9c01-616d4fec7fc0-lib-modules\") pod \"perf-node-gather-daemonset-5q7r5\" (UID: \"18fbad89-9c36-49a5-9c01-616d4fec7fc0\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.548342 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.548307    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6t6p\" (UniqueName: \"kubernetes.io/projected/18fbad89-9c36-49a5-9c01-616d4fec7fc0-kube-api-access-h6t6p\") pod \"perf-node-gather-daemonset-5q7r5\" (UID: \"18fbad89-9c36-49a5-9c01-616d4fec7fc0\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:06.628853 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:06.628819    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:07.924588 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:07.922865    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"]
Sep 11 14:06:07.924588 ip-10-0-142-132 kubenswrapper[2580]: W0911 14:06:07.923342    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod18fbad89_9c36_49a5_9c01_616d4fec7fc0.slice/crio-91deb4d59b1acacd0e4d3fabc27bc4953446ab84faca0958983fb4b024a96f93 WatchSource:0}: Error finding container 91deb4d59b1acacd0e4d3fabc27bc4953446ab84faca0958983fb4b024a96f93: Status 404 returned error can't find the container with id 91deb4d59b1acacd0e4d3fabc27bc4953446ab84faca0958983fb4b024a96f93
Sep 11 14:06:08.584727 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:08.584679    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5" event={"ID":"18fbad89-9c36-49a5-9c01-616d4fec7fc0","Type":"ContainerStarted","Data":"3abfc26c63ef8c97546f089fb9055606f87635788b74b4af22b9a7b1aadf678e"}
Sep 11 14:06:08.584951 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:08.584742    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5" event={"ID":"18fbad89-9c36-49a5-9c01-616d4fec7fc0","Type":"ContainerStarted","Data":"91deb4d59b1acacd0e4d3fabc27bc4953446ab84faca0958983fb4b024a96f93"}
Sep 11 14:06:08.584951 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:08.584768    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:09.100650 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:09.100540    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5" podStartSLOduration=3.10049794 podStartE2EDuration="3.10049794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 14:06:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:09.09898204 +0000 UTC m=+2688.317519531" watchObservedRunningTime="2026-09-11 14:06:09.10049794 +0000 UTC m=+2688.319035433"
Sep 11 14:06:09.166147 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:09.166120    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-674c9cf75d-2m2g8_bd3a03cb-d6fe-470c-b166-57c3d1a9e49d/console/0.log"
Sep 11 14:06:09.287215 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:09.287177    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-4h24k_efcfccc2-da3a-41eb-96c7-31114c25ad5d/download-server/0.log"
Sep 11 14:06:10.415178 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:10.415147    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-9xvg6_22456798-e0ae-4e71-94a9-08812435150a/volume-data-source-validator/0.log"
Sep 11 14:06:11.766853 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:11.766823    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9wxh8_934019ff-93c3-464b-906f-39c185f6d5e4/dns/0.log"
Sep 11 14:06:11.810019 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:11.809990    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9wxh8_934019ff-93c3-464b-906f-39c185f6d5e4/kube-rbac-proxy/0.log"
Sep 11 14:06:12.085458 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:12.085415    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rkrhj_030910bd-d1c8-48c5-bc44-5e561b3fbf60/dns-node-resolver/0.log"
Sep 11 14:06:12.992659 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:12.992631    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7db97697d-ndhf8_83408741-3754-4200-9226-2d58ea07355c/registry/0.log"
Sep 11 14:06:13.108194 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:13.108152    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gz6tv_c3b3d505-6502-4cc5-9aec-aef4cd996049/node-ca/0.log"
Sep 11 14:06:14.603063 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:14.603032    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-5q7r5"
Sep 11 14:06:14.669988 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:14.669958    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c9bdb7954-t6r2f_7e7c7651-a31f-413e-99d5-3e61b09e46d5/kube-auth-proxy/0.log"
Sep 11 14:06:16.118089 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:16.118044    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5zw59_fc1353d3-7bc6-45bd-9ecf-c325b4469fb8/serve-healthcheck-canary/0.log"
Sep 11 14:06:16.906932 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:16.906874    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-85bpc_09ffbcb6-3382-48ea-9c3a-04cf17da047e/kube-rbac-proxy/0.log"
Sep 11 14:06:16.941109 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:16.941078    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-85bpc_09ffbcb6-3382-48ea-9c3a-04cf17da047e/exporter/0.log"
Sep 11 14:06:16.982981 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:16.982922    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-85bpc_09ffbcb6-3382-48ea-9c3a-04cf17da047e/extractor/0.log"
Sep 11 14:06:18.905292 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:18.905258    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c9bdb7954-t6r2f_7e7c7651-a31f-413e-99d5-3e61b09e46d5/kube-auth-proxy/0.log"
Sep 11 14:06:20.639957 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:20.639897    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9576ccb85-fkn2f_4b978d74-2efb-4262-b0a0-ad66cc441e03/manager/0.log"
Sep 11 14:06:20.665049 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:20.665020    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9576ccb85-fkn2f_4b978d74-2efb-4262-b0a0-ad66cc441e03/copy-manifests/0.log"
Sep 11 14:06:21.043021 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:21.042952    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c9bdb7954-t6r2f_7e7c7651-a31f-413e-99d5-3e61b09e46d5/kube-auth-proxy/0.log"
Sep 11 14:06:21.563860 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:21.563831    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 14:06:21.576926 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:21.575708    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-mjvgv_9d5a874c-26f3-4238-91aa-70bb9c2938ae/console-operator/2.log"
Sep 11 14:06:21.996475 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:21.996395    2580 scope.go:117] "RemoveContainer" containerID="d3c48349ca9e74895d76a0f17322790995d90033f3eba03a3474178826ae82c1"
Sep 11 14:06:22.190128 ip-10-0-142-132 kubenswrapper[2580]: I0911 14:06:22.190100    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-84bdbdc8f4-kfqct_9f3cf67b-7fa7-4f98-97cf-6ba1f951cce4/manager/0.log"