Aug 05 17:17:38.493619 ip-10-0-141-37 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Aug 05 17:17:38.493631 ip-10-0-141-37 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Aug 05 17:17:38.493640 ip-10-0-141-37 systemd[1]: kubelet.service: Failed with result 'resources'.
Aug 05 17:17:38.493941 ip-10-0-141-37 systemd[1]: Failed to start Kubernetes Kubelet.
Aug 05 17:17:48.720066 ip-10-0-141-37 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Aug 05 17:17:48.720090 ip-10-0-141-37 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 0bef014f83504750a58999a294c1e474 --
Aug 05 17:19:44.896405 ip-10-0-141-37 systemd[1]: Starting Kubernetes Kubelet...
Aug 05 17:19:45.252487 ip-10-0-141-37 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 17:19:45.252487 ip-10-0-141-37 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 05 17:19:45.252487 ip-10-0-141-37 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 17:19:45.252487 ip-10-0-141-37 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 05 17:19:45.252487 ip-10-0-141-37 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 17:19:45.253363 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.253269    2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 05 17:19:45.259104 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259089    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:19:45.259104 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259103    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259107    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259111    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259114    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259118    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259121    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259123    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259127    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259130    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259148    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259151    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259154    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259162    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259165    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259168    2556 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259171    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259173    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259176    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259178    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259181    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:19:45.259180 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259184    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259187    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259189    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259193    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259196    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259198    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259201    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259203    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259206    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259208    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259211    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259213    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259216    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259218    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259221    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259223    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259226    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259228    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259231    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259234    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:19:45.259658 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259236    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259239    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259241    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259244    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259247    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259251    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259255    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259259    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259261    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259264    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259266    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259269    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259271    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259274    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259277    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259280    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259282    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259285    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259288    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:19:45.260178 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259290    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259294    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259298    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259302    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259304    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259307    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259310    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259313    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259315    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259318    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259321    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259323    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259326    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259330    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259332    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259335    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259337    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259340    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259342    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:19:45.260645 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259345    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259349    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259352    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259355    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259357    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259360    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259363    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259367    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259370    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259373    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259375    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259378    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259381    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259384    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259386    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259389    2556 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259392    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259394    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259397    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.259400    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:19:45.261100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260208    2556 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260215    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260218    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260221    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260224    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260228    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260231    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260243    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260246    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260249    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260252    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260255    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260258    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260261    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260263    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260266    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260268    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260271    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260274    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260276    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:19:45.261608 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260279    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260282    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260285    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260287    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260293    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260295    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260298    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260300    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260303    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260305    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260308    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260310    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260313    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260315    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260318    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260321    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260323    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260325    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260328    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:19:45.262100 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260330    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260338    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260341    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260345    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260348    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260351    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260354    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260357    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260360    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260363    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260365    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260368    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260371    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260375    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260378    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260380    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260383    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260386    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260388    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:19:45.262573 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260391    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260393    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260396    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260398    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260401    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260403    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260406    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260408    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260411    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260413    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260416    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260418    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260421    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260423    2556 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260427    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260437    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260440    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260442    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260445    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260447    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:19:45.263030 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260450    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260452    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260455    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260457    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260460    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260463    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260467    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260469    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260472    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260474    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260477    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260480    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260482    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260485    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260487    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260490    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260492    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260495    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260497    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:19:45.263525 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260500    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260502    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260914    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260919    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260922    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260925    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260928    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260930    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260940    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260943    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260946    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260948    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260951    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260953    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260956    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260958    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260961    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260964    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260966    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260969    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:19:45.264017 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260972    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260975    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260979    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260983    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260986    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260989    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260992    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260995    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.260997    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261000    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261003    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261006    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261010    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261013    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261016    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261019    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261022    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261025    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:19:45.264513 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261027    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261029    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261032    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261040    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261043    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261045    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261048    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261050    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261053    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261055    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261058    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261060    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261068    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261075    2556 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261077    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261080    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261082    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261085    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261087    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261090    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:19:45.264966 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261093    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261095    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261098    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261100    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261103    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261106    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261108    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261111    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261113    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261116    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261119    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261121    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261124    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261128    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261144    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261149    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261158    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261161    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261164    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261166    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:19:45.265546 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261169    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261172    2556 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261174    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261177    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261179    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261183    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261185    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261188    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261190    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261193    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261195    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261198    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261200    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261203    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261205    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261208    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261210    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261214    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261217    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:19:45.266031 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261220    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261222    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261225    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.261227    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261303    2556 flags.go:64] FLAG: --address="0.0.0.0"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261313    2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261327    2556 flags.go:64] FLAG: --anonymous-auth="true"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261332    2556 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261336    2556 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261339    2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261349    2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261354    2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261357    2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261360    2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261363    2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261367    2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261370    2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261373    2556 flags.go:64] FLAG: --cgroup-root=""
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261376    2556 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261380    2556 flags.go:64] FLAG: --client-ca-file=""
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261382    2556 flags.go:64] FLAG: --cloud-provider="external"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261385    2556 flags.go:64] FLAG: --cluster-dns="[]"
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261392    2556 flags.go:64] FLAG: --cluster-domain=""
Aug 05 17:19:45.266554 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261396    2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261399    2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261402    2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261405    2556 flags.go:64] FLAG: --container-log-max-files="5"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261409    2556 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261412    2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261415    2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261418    2556 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261421    2556 flags.go:64] FLAG: --contention-profiling="false"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261424    2556 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261427    2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261430    2556 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261433    2556 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261437    2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261440    2556 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261443    2556 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261446    2556 flags.go:64] FLAG: --enable-load-reader="false"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261450    2556 flags.go:64] FLAG: --enable-server="true"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261453    2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261460    2556 flags.go:64] FLAG: --event-burst="100"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261469    2556 flags.go:64] FLAG: --event-qps="50"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261473    2556 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261476    2556 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261478    2556 flags.go:64] FLAG: --eviction-hard=""
Aug 05 17:19:45.267112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261482    2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261485    2556 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261488    2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261491    2556 flags.go:64] FLAG: --eviction-soft=""
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261495    2556 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261497    2556 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261500    2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261503    2556 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261506    2556 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261509    2556 flags.go:64] FLAG: --fail-swap-on="true"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261512    2556 flags.go:64] FLAG: --feature-gates=""
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261516    2556 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261519    2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261521    2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261525    2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261528    2556 flags.go:64] FLAG: --healthz-port="10248"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261531    2556 flags.go:64] FLAG: --help="false"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261533    2556 flags.go:64] FLAG: --hostname-override="ip-10-0-141-37.ec2.internal"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261536    2556 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261540    2556 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261543    2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261546    2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261549    2556 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261552    2556 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 05 17:19:45.267761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261555    2556 flags.go:64] FLAG: --image-service-endpoint=""
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261559    2556 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261561    2556 flags.go:64] FLAG: --kube-api-burst="100"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261564    2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261568    2556 flags.go:64] FLAG: --kube-api-qps="50"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261576    2556 flags.go:64] FLAG: --kube-reserved=""
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261579    2556 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261582    2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261585    2556 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261587    2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261590    2556 flags.go:64] FLAG: --lock-file=""
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261593    2556 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261596    2556 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261599    2556 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261604    2556 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261607    2556 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261610    2556 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261613    2556 flags.go:64] FLAG: --logging-format="text"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261616    2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261620    2556 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261623    2556 flags.go:64] FLAG: --manifest-url=""
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261626    2556 flags.go:64] FLAG: --manifest-url-header=""
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261630    2556 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261633    2556 flags.go:64] FLAG: --max-open-files="1000000"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261637    2556 flags.go:64] FLAG: --max-pods="110"
Aug 05 17:19:45.268661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261640    2556 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261643    2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261645    2556 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261648    2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261651    2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261654    2556 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261657    2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261664    2556 flags.go:64] FLAG: --node-status-max-images="50"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261668    2556 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261671    2556 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261674    2556 flags.go:64] FLAG: --pod-cidr=""
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261677    2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261685    2556 flags.go:64] FLAG: --pod-manifest-path=""
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261694    2556 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261698    2556 flags.go:64] FLAG: --pods-per-core="0"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261701    2556 flags.go:64] FLAG: --port="10250"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261703    2556 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261706    2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0daeac400c17c3b3e"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261709    2556 flags.go:64] FLAG: --qos-reserved=""
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261713    2556 flags.go:64] FLAG: --read-only-port="10255"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261716    2556 flags.go:64] FLAG: --register-node="true"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261719    2556 flags.go:64] FLAG: --register-with-taints=""
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261723    2556 flags.go:64] FLAG: --registry-burst="10"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261726    2556 flags.go:64] FLAG: --registry-qps="5"
Aug 05 17:19:45.269636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261729    2556 flags.go:64] FLAG: --reserved-cpus=""
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261732    2556 flags.go:64] FLAG: --reserved-memory=""
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261735    2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261738    2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261741    2556 flags.go:64] FLAG: --rotate-certificates="false"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261744    2556 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261747    2556 flags.go:64] FLAG: --runonce="false"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261750    2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261753    2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261755    2556 flags.go:64] FLAG: --seccomp-default="false"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261758    2556 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261761    2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261764    2556 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261767    2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261770    2556 flags.go:64] FLAG: --storage-driver-password="root"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261772    2556 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261777    2556 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261779    2556 flags.go:64] FLAG: --storage-driver-user="root"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261782    2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261785    2556 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261788    2556 flags.go:64] FLAG: --system-cgroups=""
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261791    2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261802    2556 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261805    2556 flags.go:64] FLAG: --tls-cert-file=""
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261807    2556 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 05 17:19:45.270304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261814    2556 flags.go:64] FLAG: --tls-min-version=""
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261817    2556 flags.go:64] FLAG: --tls-private-key-file=""
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261820    2556 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261823    2556 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261826    2556 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261829    2556 flags.go:64] FLAG: --v="2"
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261833    2556 flags.go:64] FLAG: --version="false"
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261837    2556 flags.go:64] FLAG: --vmodule=""
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261841    2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.261844    2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262004    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262008    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262010    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262013    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262016    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262018    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262021    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262023    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262026    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262028    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262031    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262033    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:19:45.270952 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262036    2556 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262040    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262043    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262046    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262048    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262050    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262053    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262055    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262065    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262068    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262071    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262073    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262078    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262082    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262085    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262088    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262091    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262094    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262097    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:19:45.271574 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262100    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262103    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262107    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262110    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262112    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262115    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262118    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262120    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262123    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262127    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262130    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262153    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262156    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262158    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262162    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262166    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262170    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262173    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262176    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:19:45.272038 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262178    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262181    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262184    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262193    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262195    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262198    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262202    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262204    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262207    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262210    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262213    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262215    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262218    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262220    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262223    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262225    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262228    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262231    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262233    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262236    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:19:45.272518 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262238    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262241    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262244    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262246    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262248    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262251    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262253    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262257    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262260    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262262    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262265    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262267    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262270    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262272    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262275    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262277    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262285    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262288    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262292    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262294    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:19:45.272997 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262297    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:19:45.273491 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262299    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:19:45.273491 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262302    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:19:45.273491 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262305    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:19:45.273491 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262310    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:19:45.273491 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262313    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:19:45.273491 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262315    2556 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:19:45.273491 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262318    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:19:45.273491 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.262320    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:19:45.273491 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.262329    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 05 17:19:45.273491 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.269998    2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 05 17:19:45.273491 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.270012    2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 05 17:19:45.273491 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.270736    2556 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 05 17:19:45.273491 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.270748    2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 05 17:19:45.273491 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270803    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270807    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270811    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270814    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270817    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270821    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270824    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270827    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270830    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270833    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270836    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270840    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270844    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270847    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270850    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270853    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270856    2556 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270858    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270861    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270864    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:19:45.273835 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270867    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270870    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270872    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270875    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270877    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270880    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270883    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270885    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270888    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270890    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270893    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270896    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270898    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270901    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270903    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270906    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270908    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270911    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270915    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:19:45.274337 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270918    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270921    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270924    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270927    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270929    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270932    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270935    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270938    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270940    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270943    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270946    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270949    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270952    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270955    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270958    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270961    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270964    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270966    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270969    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270971    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:19:45.274865 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270975    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270977    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270980    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270982    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270985    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270987    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270989    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270992    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270995    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.270997    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271000    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271002    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271006    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271010    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271013    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271015    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271018    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271021    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271023    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:19:45.275447 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271026    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271029    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271031    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271034    2556 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271037    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271039    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271042    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271045    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271047    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271049    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271052    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271055    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271057    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271060    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271062    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271065    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271067    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271069    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271072    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271075    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:19:45.275921 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271077    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:19:45.276530 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.271082    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 05 17:19:45.276530 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271212    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:19:45.276530 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271218    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:19:45.276530 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271222    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:19:45.276530 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271225    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:19:45.276530 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271228    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:19:45.276530 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271230    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:19:45.276530 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271233    2556 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:19:45.276530 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271236    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:19:45.276530 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271239    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:19:45.276530 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271241    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:19:45.276530 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271244    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:19:45.276530 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271247    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:19:45.276530 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271250    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:19:45.276530 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271253    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271256    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271258    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271261    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271264    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271266    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271269    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271271    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271278    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271281    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271283    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271287    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271291    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271294    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271297    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271299    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271302    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271305    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271308    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:19:45.276895 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271311    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271313    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271316    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271318    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271321    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271323    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271326    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271329    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271331    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271334    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271336    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271339    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271341    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271344    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271347    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271351    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271354    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271357    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271360    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271363    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:19:45.277632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271366    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271368    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271371    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271373    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271376    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271378    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271381    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271383    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271386    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271388    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271391    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271394    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271396    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271399    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271401    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271404    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271406    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271409    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271421    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:19:45.278403 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271424    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271427    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271430    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271432    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271434    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271437    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271439    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271445    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271448    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271450    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271453    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271455    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271458    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271461    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271464    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271466    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271470    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271473    2556 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271475    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271478    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:19:45.278864 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271480    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:19:45.279679 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271483    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:19:45.279679 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271485    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:19:45.279679 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271488    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:19:45.279679 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271491    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:19:45.279679 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271493    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:19:45.279679 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271495    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:19:45.279679 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:45.271498    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:19:45.279679 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.271503    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 05 17:19:45.279679 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.271601    2556 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 05 17:19:45.279679 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.274455    2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 05 17:19:45.279679 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.275905    2556 server.go:1019] "Starting client certificate rotation"
Aug 05 17:19:45.279679 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.276026    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 05 17:19:45.279679 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.276077    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 05 17:19:45.279679 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.278240    2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 05 17:19:45.280099 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.279358    2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 05 17:19:45.288586 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.288569    2556 log.go:25] "Validated CRI v1 runtime API"
Aug 05 17:19:45.293062 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.293047    2556 log.go:25] "Validated CRI v1 image API"
Aug 05 17:19:45.294599 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.294583    2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 05 17:19:45.296720 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.296702    2556 fs.go:135] Filesystem UUIDs: map[63789181-bd19-4458-b980-4f930602f10c:/dev/nvme0n1p3 71ceac03-54ff-40b3-b4e1-9f1034c721d7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Aug 05 17:19:45.296770 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.296720    2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 05 17:19:45.301292 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.301197    2556 manager.go:216] Machine: {Timestamp:2026-08-05 17:19:45.300733106 +0000 UTC m=+0.299063759 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100731 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2310cbc3b81d929822ff2f8ebf81e2 SystemUUID:ec2310cb-c3b8-1d92-9822-ff2f8ebf81e2 BootID:0bef014f-8350-4750-a589-99a294c1e474 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d1:51:5c:93:79 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d1:51:5c:93:79 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:d4:ee:f5:5d:64 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 05 17:19:45.301292 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.301289    2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 05 17:19:45.301404 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.301354    2556 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 05 17:19:45.304874 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.304851    2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 05 17:19:45.306020 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.304877    2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-37.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 05 17:19:45.306291 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.306196    2556 topology_manager.go:138] "Creating topology manager with none policy"
Aug 05 17:19:45.306384 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.306372    2556 container_manager_linux.go:309] "Creating device plugin manager"
Aug 05 17:19:45.306448 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.306391    2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 05 17:19:45.306563 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.306539    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 05 17:19:45.306970 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.306954    2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 05 17:19:45.308209 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.308193    2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 05 17:19:45.311964 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.311948    2556 state_mem.go:36] "Initialized new in-memory state store"
Aug 05 17:19:45.312119 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.312107    2556 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 05 17:19:45.312206 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.312161    2556 kubelet.go:486] "Attempting to sync node with API server"
Aug 05 17:19:45.312206 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.312177    2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 05 17:19:45.312206 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.312191    2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 05 17:19:45.312206 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.312204    2556 kubelet.go:398] "Adding apiserver pod source"
Aug 05 17:19:45.312360 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.312214    2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 05 17:19:45.312577 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.312560    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 05 17:19:45.312667 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.312595    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 05 17:19:45.315403 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.315389    2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 05 17:19:45.316942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.316930    2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 05 17:19:45.316981 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.316958    2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 05 17:19:45.318204 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.318192    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 05 17:19:45.318263 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.318208    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 05 17:19:45.318263 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.318214    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 05 17:19:45.318263 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.318219    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 05 17:19:45.318263 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.318226    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 05 17:19:45.318263 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.318232    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 05 17:19:45.318263 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.318239    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 05 17:19:45.318263 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.318244    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 05 17:19:45.318263 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.318251    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 05 17:19:45.318263 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.318257    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 05 17:19:45.318263 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.318265    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 05 17:19:45.318518 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.318270    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 05 17:19:45.318864 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.318856    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 05 17:19:45.318895 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.318865    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 05 17:19:45.321784 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.321771    2556 server.go:1268] "Started kubelet"
Aug 05 17:19:45.321864 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.321840    2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 05 17:19:45.322048 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.321996    2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 05 17:19:45.322158 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.322055    2556 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 05 17:19:45.322568 ip-10-0-141-37 systemd[1]: Started Kubernetes Kubelet.
Aug 05 17:19:45.322722 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:45.322705    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 05 17:19:45.322760 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:45.322706    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 05 17:19:45.322760 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.322731    2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-37.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 05 17:19:45.323585 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.323522    2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 05 17:19:45.324280 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.324267    2556 server.go:310] "Adding debug handlers to kubelet server"
Aug 05 17:19:45.327238 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:45.327221    2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 05 17:19:45.327337 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:45.326529    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-37.ec2.internal.18c8f869bf93141e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-37.ec2.internal,UID:ip-10-0-141-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-37.ec2.internal,},FirstTimestamp:2026-08-05 17:19:45.32174339 +0000 UTC m=+0.320074024,LastTimestamp:2026-08-05 17:19:45.32174339 +0000 UTC m=+0.320074024,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-37.ec2.internal,}"
Aug 05 17:19:45.329538 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.329521    2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 05 17:19:45.329630 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.329542    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 05 17:19:45.329689 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:45.329646    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found"
Aug 05 17:19:45.330234 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.330219    2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 05 17:19:45.330234 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.330228    2556 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 05 17:19:45.330350 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.330240    2556 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 05 17:19:45.330350 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.330344    2556 reconstruct.go:97] "Volume reconstruction finished"
Aug 05 17:19:45.330445 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.330354    2556 reconciler.go:29] "Reconciler: start to sync state"
Aug 05 17:19:45.330732 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.330716    2556 factory.go:153] Registering CRI-O factory
Aug 05 17:19:45.330732 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.330734    2556 factory.go:223] Registration of the crio container factory successfully
Aug 05 17:19:45.330840 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.330776    2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 05 17:19:45.330840 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.330784    2556 factory.go:55] Registering systemd factory
Aug 05 17:19:45.330840 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.330789    2556 factory.go:223] Registration of the systemd container factory successfully
Aug 05 17:19:45.330840 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.330804    2556 factory.go:103] Registering Raw factory
Aug 05 17:19:45.330840 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.330812    2556 manager.go:1196] Started watching for new ooms in manager
Aug 05 17:19:45.331318 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.331302    2556 manager.go:349] Starting recovery of all containers
Aug 05 17:19:45.331733 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:45.331528    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Aug 05 17:19:45.331793 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:45.331731    2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Aug 05 17:19:45.333860 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.333836    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tjtqc"
Aug 05 17:19:45.342348 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.342326    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tjtqc"
Aug 05 17:19:45.343472 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.343456    2556 manager.go:354] Recovery completed
Aug 05 17:19:45.347389 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.347376    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:19:45.349173 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.349156    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:19:45.349240 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.349185    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:19:45.349240 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.349194    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:19:45.349658 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.349644    2556 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 05 17:19:45.349658 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.349655    2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 05 17:19:45.349658 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.349656    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:19:45.349795 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.349672    2556 state_mem.go:36] "Initialized new in-memory state store"
Aug 05 17:19:45.350317 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.350301    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:19:45.350390 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.350326    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:19:45.350390 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.350338    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:19:45.350701 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.350691    2556 policy_none.go:49] "None policy: Start"
Aug 05 17:19:45.350732 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.350705    2556 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 05 17:19:45.350732 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.350715    2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 05 17:19:45.351658 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.351645    2556 policy_none.go:47] "Start"
Aug 05 17:19:45.386770 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.386659    2556 manager.go:337] "Starting Device Plugin manager"
Aug 05 17:19:45.386858 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:45.386795    2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 05 17:19:45.386858 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.386809    2556 server.go:87] "Starting device plugin registration server"
Aug 05 17:19:45.387040 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.387027    2556 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 05 17:19:45.387102 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.387041    2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 05 17:19:45.387167 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.387129    2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 05 17:19:45.387223 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.387216    2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 05 17:19:45.387272 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.387225    2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 05 17:19:45.387692 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:45.387663    2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 05 17:19:45.387770 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:45.387723    2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-37.ec2.internal\" not found"
Aug 05 17:19:45.487603 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.487578    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:19:45.488923 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.488906    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:19:45.488988 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.488936    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:19:45.488988 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.488955    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:19:45.488988 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.488987    2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-37.ec2.internal"
Aug 05 17:19:45.497667 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.497650    2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-37.ec2.internal"
Aug 05 17:19:45.497718 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:45.497670    2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-37.ec2.internal\": node \"ip-10-0-141-37.ec2.internal\" not found"
Aug 05 17:19:45.509162 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.509098    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 05 17:19:45.510287 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.510267    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 05 17:19:45.510362 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.510311    2556 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 05 17:19:45.510362 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.510335    2556 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 05 17:19:45.510444 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:45.510372    2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 05 17:19:45.512953 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.512937    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:19:45.516099 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:45.516078    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found"
Aug 05 17:19:45.610986 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.610954    2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal"]
Aug 05 17:19:45.611078 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.611044    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:19:45.613442 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.613428    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:19:45.613494 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.613459    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:19:45.613494 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.613474    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:19:45.614806 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.614795    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:19:45.614926 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.614910    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal"
Aug 05 17:19:45.614972 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.614964    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:19:45.616166 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:45.616130    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found"
Aug 05 17:19:45.616269 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.616256    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:19:45.616313 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.616289    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:19:45.616313 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.616301    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:19:45.616378 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.616290    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:19:45.616414 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.616393    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:19:45.616414 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.616406    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:19:45.617584 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.617570    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal"
Aug 05 17:19:45.617638 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.617599    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:19:45.618274 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.618262    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:19:45.618325 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.618281    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:19:45.618325 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:45.618318    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:19:47.211855 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.211561    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/29384c2fdf1ce8bc484da2b88a670c44-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal\" (UID: \"29384c2fdf1ce8bc484da2b88a670c44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal"
Aug 05 17:19:47.211855 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.211589    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29384c2fdf1ce8bc484da2b88a670c44-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal\" (UID: \"29384c2fdf1ce8bc484da2b88a670c44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal"
Aug 05 17:19:47.211855 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.211611    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/13b14f86a6dd04037249bf3e254a8bd2-config\") pod \"kube-apiserver-proxy-ip-10-0-141-37.ec2.internal\" (UID: \"13b14f86a6dd04037249bf3e254a8bd2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal"
Aug 05 17:19:47.229155 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:47.229130    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-37.ec2.internal\" not found" node="ip-10-0-141-37.ec2.internal"
Aug 05 17:19:47.233529 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:47.233510    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-37.ec2.internal\" not found" node="ip-10-0-141-37.ec2.internal"
Aug 05 17:19:47.296323 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:47.296301    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found"
Aug 05 17:19:47.311718 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.311690    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/29384c2fdf1ce8bc484da2b88a670c44-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal\" (UID: \"29384c2fdf1ce8bc484da2b88a670c44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal"
Aug 05 17:19:47.311788 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.311722    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29384c2fdf1ce8bc484da2b88a670c44-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal\" (UID: \"29384c2fdf1ce8bc484da2b88a670c44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal"
Aug 05 17:19:47.311788 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.311741    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/13b14f86a6dd04037249bf3e254a8bd2-config\") pod \"kube-apiserver-proxy-ip-10-0-141-37.ec2.internal\" (UID: \"13b14f86a6dd04037249bf3e254a8bd2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal"
Aug 05 17:19:47.311788 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.311772    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29384c2fdf1ce8bc484da2b88a670c44-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal\" (UID: \"29384c2fdf1ce8bc484da2b88a670c44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal"
Aug 05 17:19:47.311788 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.311780    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/29384c2fdf1ce8bc484da2b88a670c44-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal\" (UID: \"29384c2fdf1ce8bc484da2b88a670c44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal"
Aug 05 17:19:47.311922 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.311816    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/13b14f86a6dd04037249bf3e254a8bd2-config\") pod \"kube-apiserver-proxy-ip-10-0-141-37.ec2.internal\" (UID: \"13b14f86a6dd04037249bf3e254a8bd2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal"
Aug 05 17:19:47.396529 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:47.396465    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found"
Aug 05 17:19:47.496923 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:47.496900    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found"
Aug 05 17:19:47.532435 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.532416    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal"
Aug 05 17:19:47.537103 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.536878    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal"
Aug 05 17:19:47.597741 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:47.597710    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found"
Aug 05 17:19:47.698200 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:47.698122    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found"
Aug 05 17:19:47.798659 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:47.798626    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found"
Aug 05 17:19:47.855146 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.855122    2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 05 17:19:47.855573 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.855274    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 05 17:19:47.899528 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:47.899494    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-37.ec2.internal\" not found"
Aug 05 17:19:47.914638 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.914614    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:19:47.926926 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.926900    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-04 17:14:46 +0000 UTC" deadline="2028-05-03 06:16:30.613463194 +0000 UTC"
Aug 05 17:19:47.926926 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.926922    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15276h56m42.68654429s"
Aug 05 17:19:47.979612 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:47.979597    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:19:48.009982 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.009963    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal"
Aug 05 17:19:48.022039 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.022021    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 05 17:19:48.024344 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.024323    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal"
Aug 05 17:19:48.032706 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.032691    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 05 17:19:48.061018 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.060996    2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:19:48.073431 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:48.073397    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13b14f86a6dd04037249bf3e254a8bd2.slice/crio-36e65d325e0a5e5f67feb8eeb189d3ffb5bbf292ae20e62598eb7755619f66e1 WatchSource:0}: Error finding container 36e65d325e0a5e5f67feb8eeb189d3ffb5bbf292ae20e62598eb7755619f66e1: Status 404 returned error can't find the container with id 36e65d325e0a5e5f67feb8eeb189d3ffb5bbf292ae20e62598eb7755619f66e1
Aug 05 17:19:48.073617 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:48.073604    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29384c2fdf1ce8bc484da2b88a670c44.slice/crio-67d9f2fa3203453959baa62ef778fe07d9c8da4ef81e8a70119ec2d8e0019552 WatchSource:0}: Error finding container 67d9f2fa3203453959baa62ef778fe07d9c8da4ef81e8a70119ec2d8e0019552: Status 404 returned error can't find the container with id 67d9f2fa3203453959baa62ef778fe07d9c8da4ef81e8a70119ec2d8e0019552
Aug 05 17:19:48.078714 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.078693    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:19:48.094144 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.094100    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal" event={"ID":"13b14f86a6dd04037249bf3e254a8bd2","Type":"ContainerStarted","Data":"36e65d325e0a5e5f67feb8eeb189d3ffb5bbf292ae20e62598eb7755619f66e1"}
Aug 05 17:19:48.095540 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.095518    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" event={"ID":"29384c2fdf1ce8bc484da2b88a670c44","Type":"ContainerStarted","Data":"67d9f2fa3203453959baa62ef778fe07d9c8da4ef81e8a70119ec2d8e0019552"}
Aug 05 17:19:48.894268 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.894230    2556 apiserver.go:52] "Watching apiserver"
Aug 05 17:19:48.902464 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.902443    2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 05 17:19:48.903557 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.903527    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-rzlln","openshift-multus/network-metrics-daemon-8fkfq","openshift-network-operator/iptables-alerter-k2tgt","kube-system/global-pull-secret-syncer-t2vbt","kube-system/konnectivity-agent-q4cgr","kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal","openshift-dns/node-resolver-5z6td","openshift-image-registry/node-ca-l2prc","openshift-multus/multus-74dt8","openshift-network-diagnostics/network-check-target-5zfsf","openshift-ovn-kubernetes/ovnkube-node-l9ptq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7","openshift-cluster-node-tuning-operator/tuned-pgfr7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal"]
Aug 05 17:19:48.904842 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.904818    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5z6td"
Aug 05 17:19:48.906645 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.906618    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:48.907593 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.907266    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 05 17:19:48.907593 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.907287    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 05 17:19:48.907593 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.907455    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zvpwb\""
Aug 05 17:19:48.908225 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.908202    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k2tgt"
Aug 05 17:19:48.908877 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.908858    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:19:48.909065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.909040    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-d9zh9\""
Aug 05 17:19:48.909337 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.909318    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 05 17:19:48.909518 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.909501    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:48.909599 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:48.909577    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:19:48.909806 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.909772    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 05 17:19:48.910886 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.910865    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:19:48.910978 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.910924    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c9vsw\""
Aug 05 17:19:48.911053 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.911036    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 05 17:19:48.911121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.911107    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 05 17:19:48.911251 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.911235    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q4cgr"
Aug 05 17:19:48.912971 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.912497    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l2prc"
Aug 05 17:19:48.913388 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.913370    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-74dt8"
Aug 05 17:19:48.913980 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.913961    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 05 17:19:48.914067 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.913986    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5dt47\""
Aug 05 17:19:48.914128 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.913964    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 05 17:19:48.914789 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.914767    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:48.914868 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.914837    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 05 17:19:48.914868 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.914852    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dzc6c\""
Aug 05 17:19:48.915396 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.915355    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 05 17:19:48.915510 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.915408    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 05 17:19:48.915596 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.915576    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 05 17:19:48.916095 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.916077    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 05 17:19:48.916404 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.916376    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:19:48.916497 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.916425    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-s5slm\""
Aug 05 17:19:48.916497 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:48.916445    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:19:48.916608 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.916509    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 05 17:19:48.916657 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.916626    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 05 17:19:48.917511 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.917491    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-584xh\""
Aug 05 17:19:48.917630 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.917612    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:19:48.917698 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:48.917676    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:19:48.917816 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.917794    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 05 17:19:48.920743 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.920722    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-var-lib-kubelet\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:48.920817 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.920765    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-tuned\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:48.920817 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.920797    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-tmp\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:48.920901 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.920823    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-systemd\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:48.920901 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.920854    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-run\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:48.920901 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.920883    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-host\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:48.920998 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.920914    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jls2l\" (UniqueName: \"kubernetes.io/projected/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-kube-api-access-jls2l\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:48.920998 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.920957    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c2873311-84df-4d01-ac08-3deb1f12420c-iptables-alerter-script\") pod \"iptables-alerter-k2tgt\" (UID: \"c2873311-84df-4d01-ac08-3deb1f12420c\") " pod="openshift-network-operator/iptables-alerter-k2tgt"
Aug 05 17:19:48.920998 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.920990    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c2873311-84df-4d01-ac08-3deb1f12420c-host-slash\") pod \"iptables-alerter-k2tgt\" (UID: \"c2873311-84df-4d01-ac08-3deb1f12420c\") " pod="openshift-network-operator/iptables-alerter-k2tgt"
Aug 05 17:19:48.921132 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.921022    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rspm\" (UniqueName: \"kubernetes.io/projected/c2873311-84df-4d01-ac08-3deb1f12420c-kube-api-access-5rspm\") pod \"iptables-alerter-k2tgt\" (UID: \"c2873311-84df-4d01-ac08-3deb1f12420c\") " pod="openshift-network-operator/iptables-alerter-k2tgt"
Aug 05 17:19:48.921132 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.921053    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46b91743-4ee2-4ac1-9779-5b85586520ec-kubelet-config\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:48.921132 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.921087    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e8d949ff-bbbc-4247-87f0-d9f98e7ec389-hosts-file\") pod \"node-resolver-5z6td\" (UID: \"e8d949ff-bbbc-4247-87f0-d9f98e7ec389\") " pod="openshift-dns/node-resolver-5z6td"
Aug 05 17:19:48.921132 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.921117    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-sysconfig\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:48.921296 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.921141    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-sysctl-d\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:48.921296 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.921207    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-sysctl-conf\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:48.921296 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.921235    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-sys\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:48.921296 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.921267    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46b91743-4ee2-4ac1-9779-5b85586520ec-dbus\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:48.921494 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.921298    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6489ed96-823a-4ff5-bab4-0efabcb3584b-agent-certs\") pod \"konnectivity-agent-q4cgr\" (UID: \"6489ed96-823a-4ff5-bab4-0efabcb3584b\") " pod="kube-system/konnectivity-agent-q4cgr"
Aug 05 17:19:48.921601 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.921581    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 05 17:19:48.922164 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.922120    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8d949ff-bbbc-4247-87f0-d9f98e7ec389-tmp-dir\") pod \"node-resolver-5z6td\" (UID: \"e8d949ff-bbbc-4247-87f0-d9f98e7ec389\") " pod="openshift-dns/node-resolver-5z6td"
Aug 05 17:19:48.922295 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.922196    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-lib-modules\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:48.922415 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.922300    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:48.922415 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.922343    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6489ed96-823a-4ff5-bab4-0efabcb3584b-konnectivity-ca\") pod \"konnectivity-agent-q4cgr\" (UID: \"6489ed96-823a-4ff5-bab4-0efabcb3584b\") " pod="kube-system/konnectivity-agent-q4cgr"
Aug 05 17:19:48.922415 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.922383    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh96h\" (UniqueName: \"kubernetes.io/projected/e8d949ff-bbbc-4247-87f0-d9f98e7ec389-kube-api-access-nh96h\") pod \"node-resolver-5z6td\" (UID: \"e8d949ff-bbbc-4247-87f0-d9f98e7ec389\") " pod="openshift-dns/node-resolver-5z6td"
Aug 05 17:19:48.922591 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.922419    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-modprobe-d\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:48.922591 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.922546    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-kubernetes\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:48.923035 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.922968    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:48.924657 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.924637    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:48.925644 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.925377    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 05 17:19:48.926212 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.926191    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xpz8w\""
Aug 05 17:19:48.926329 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.926311    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 05 17:19:48.926594 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.926316    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 05 17:19:48.926675 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.926383    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 05 17:19:48.927274 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.927257    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 05 17:19:48.927589 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.927397    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 05 17:19:48.927589 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.927524    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 05 17:19:48.927743 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.927696    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 05 17:19:48.927743 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.927737    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 05 17:19:48.928678 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.928657    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x2jbd\""
Aug 05 17:19:48.932221 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.932200    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 05 17:19:48.945471 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.945449    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gmkk4"
Aug 05 17:19:48.953960 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:48.953815    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gmkk4"
Aug 05 17:19:49.011682 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.011664    2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 05 17:19:49.023019 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.022958    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-multus-cni-dir\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.023019 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.022996    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4d033317-735c-4582-b54f-f63c0e0663fb-cnibin\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.023185 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023030    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4d033317-735c-4582-b54f-f63c0e0663fb-os-release\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.023185 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023098    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-log-socket\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.023185 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023169    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/658875a2-a766-417a-b50e-ee46f5ba179b-ovn-node-metrics-cert\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.023391 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023204    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-kubernetes\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.023391 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023235    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-var-lib-kubelet\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.023391 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023259    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jls2l\" (UniqueName: \"kubernetes.io/projected/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-kube-api-access-jls2l\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.023391 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023317    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-registration-dir\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.023391 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023336    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-var-lib-kubelet\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.023391 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023344    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zgdp\" (UniqueName: \"kubernetes.io/projected/29967c90-6d3d-43b1-a472-714f10ab92ab-kube-api-access-4zgdp\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.023391 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023355    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-kubernetes\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.023391 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023372    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-slash\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.023758 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023404    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c2873311-84df-4d01-ac08-3deb1f12420c-iptables-alerter-script\") pod \"iptables-alerter-k2tgt\" (UID: \"c2873311-84df-4d01-ac08-3deb1f12420c\") " pod="openshift-network-operator/iptables-alerter-k2tgt"
Aug 05 17:19:49.023758 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023450    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.023758 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023554    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-socket-dir\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.023758 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023586    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-var-lib-openvswitch\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.023758 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023615    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-node-log\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.023758 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023657    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e8d949ff-bbbc-4247-87f0-d9f98e7ec389-hosts-file\") pod \"node-resolver-5z6td\" (UID: \"e8d949ff-bbbc-4247-87f0-d9f98e7ec389\") " pod="openshift-dns/node-resolver-5z6td"
Aug 05 17:19:49.023758 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023699    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-sys\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.023758 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023717    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e8d949ff-bbbc-4247-87f0-d9f98e7ec389-hosts-file\") pod \"node-resolver-5z6td\" (UID: \"e8d949ff-bbbc-4247-87f0-d9f98e7ec389\") " pod="openshift-dns/node-resolver-5z6td"
Aug 05 17:19:49.023758 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023727    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-run-multus-certs\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.023758 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023752    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-run-ovn-kubernetes\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023769    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-sys\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023777    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-cni-netd\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023802    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8d949ff-bbbc-4247-87f0-d9f98e7ec389-tmp-dir\") pod \"node-resolver-5z6td\" (UID: \"e8d949ff-bbbc-4247-87f0-d9f98e7ec389\") " pod="openshift-dns/node-resolver-5z6td"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023826    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-lib-modules\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023850    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-var-lib-cni-multus\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023848    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c2873311-84df-4d01-ac08-3deb1f12420c-iptables-alerter-script\") pod \"iptables-alerter-k2tgt\" (UID: \"c2873311-84df-4d01-ac08-3deb1f12420c\") " pod="openshift-network-operator/iptables-alerter-k2tgt"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023877    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-run-systemd\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023902    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/658875a2-a766-417a-b50e-ee46f5ba179b-ovnkube-config\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023938    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46b91743-4ee2-4ac1-9779-5b85586520ec-kubelet-config\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023961    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-sys-fs\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.023984    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-var-lib-kubelet\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024006    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-hostroot\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024027    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-multus-conf-dir\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024027    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-lib-modules\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024100    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46b91743-4ee2-4ac1-9779-5b85586520ec-kubelet-config\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024098    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8d949ff-bbbc-4247-87f0-d9f98e7ec389-tmp-dir\") pod \"node-resolver-5z6td\" (UID: \"e8d949ff-bbbc-4247-87f0-d9f98e7ec389\") " pod="openshift-dns/node-resolver-5z6td"
Aug 05 17:19:49.024267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024118    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-kubelet\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024143    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-etc-openvswitch\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024196    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-host\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024218    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a264b228-b7f6-4977-b858-820fc604b53c-host\") pod \"node-ca-l2prc\" (UID: \"a264b228-b7f6-4977-b858-820fc604b53c\") " pod="openshift-image-registry/node-ca-l2prc"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024241    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-os-release\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024300    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-systemd-units\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024325    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-run-openvswitch\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024351    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6489ed96-823a-4ff5-bab4-0efabcb3584b-konnectivity-ca\") pod \"konnectivity-agent-q4cgr\" (UID: \"6489ed96-823a-4ff5-bab4-0efabcb3584b\") " pod="kube-system/konnectivity-agent-q4cgr"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024375    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-device-dir\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024399    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqdc7\" (UniqueName: \"kubernetes.io/projected/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-kube-api-access-jqdc7\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024443    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4d033317-735c-4582-b54f-f63c0e0663fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024474    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a264b228-b7f6-4977-b858-820fc604b53c-serviceca\") pod \"node-ca-l2prc\" (UID: \"a264b228-b7f6-4977-b858-820fc604b53c\") " pod="openshift-image-registry/node-ca-l2prc"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024499    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5qjt\" (UniqueName: \"kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt\") pod \"network-check-target-5zfsf\" (UID: \"e9766cbc-f4f7-402d-9f9b-15e132d49b71\") " pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024523    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-cni-bin\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024562    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpn46\" (UniqueName: \"kubernetes.io/projected/658875a2-a766-417a-b50e-ee46f5ba179b-kube-api-access-bpn46\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024610    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh96h\" (UniqueName: \"kubernetes.io/projected/e8d949ff-bbbc-4247-87f0-d9f98e7ec389-kube-api-access-nh96h\") pod \"node-resolver-5z6td\" (UID: \"e8d949ff-bbbc-4247-87f0-d9f98e7ec389\") " pod="openshift-dns/node-resolver-5z6td"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024634    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-tuned\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.025065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024658    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rspm\" (UniqueName: \"kubernetes.io/projected/c2873311-84df-4d01-ac08-3deb1f12420c-kube-api-access-5rspm\") pod \"iptables-alerter-k2tgt\" (UID: \"c2873311-84df-4d01-ac08-3deb1f12420c\") " pod="openshift-network-operator/iptables-alerter-k2tgt"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024694    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4d033317-735c-4582-b54f-f63c0e0663fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024717    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs\") pod \"network-metrics-daemon-8fkfq\" (UID: \"bfd52bcf-e224-4a4c-8fa2-13c94f985e60\") " pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024739    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/658875a2-a766-417a-b50e-ee46f5ba179b-env-overrides\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024772    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-systemd\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024797    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024839    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj8bt\" (UniqueName: \"kubernetes.io/projected/a264b228-b7f6-4977-b858-820fc604b53c-kube-api-access-fj8bt\") pod \"node-ca-l2prc\" (UID: \"a264b228-b7f6-4977-b858-820fc604b53c\") " pod="openshift-image-registry/node-ca-l2prc"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024884    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-sysconfig\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024921    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-sysctl-d\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024944    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-sysctl-conf\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.024967    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-cnibin\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025000    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-multus-socket-dir-parent\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025023    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-run-ovn\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025052    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/658875a2-a766-417a-b50e-ee46f5ba179b-ovnkube-script-lib\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025075    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-cni-binary-copy\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025098    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-run-k8s-cni-cncf-io\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025118    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-host\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.025850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025123    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-var-lib-cni-bin\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025165    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-multus-daemon-config\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025189    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-run-netns\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025212    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-etc-kubernetes\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025263    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-modprobe-d\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025290    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-tmp\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025318    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c2873311-84df-4d01-ac08-3deb1f12420c-host-slash\") pod \"iptables-alerter-k2tgt\" (UID: \"c2873311-84df-4d01-ac08-3deb1f12420c\") " pod="openshift-network-operator/iptables-alerter-k2tgt"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025344    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-run-netns\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025371    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025402    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-run\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025442    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-system-cni-dir\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025466    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4d033317-735c-4582-b54f-f63c0e0663fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025488    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46b91743-4ee2-4ac1-9779-5b85586520ec-dbus\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025510    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6489ed96-823a-4ff5-bab4-0efabcb3584b-agent-certs\") pod \"konnectivity-agent-q4cgr\" (UID: \"6489ed96-823a-4ff5-bab4-0efabcb3584b\") " pod="kube-system/konnectivity-agent-q4cgr"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025524    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-modprobe-d\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025546    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d033317-735c-4582-b54f-f63c0e0663fb-system-cni-dir\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025268    2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 05 17:19:49.026741 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.025690    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-sysctl-conf\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.027509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.026021    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-sysconfig\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.027509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.026049    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-run\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.027509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.026069    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-systemd\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.027509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.026183    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-sysctl-d\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.027509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.026189    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46b91743-4ee2-4ac1-9779-5b85586520ec-dbus\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:49.027509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.026227    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c2873311-84df-4d01-ac08-3deb1f12420c-host-slash\") pod \"iptables-alerter-k2tgt\" (UID: \"c2873311-84df-4d01-ac08-3deb1f12420c\") " pod="openshift-network-operator/iptables-alerter-k2tgt"
Aug 05 17:19:49.027509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.026266    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4d033317-735c-4582-b54f-f63c0e0663fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.027509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.026351    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7lhz\" (UniqueName: \"kubernetes.io/projected/4d033317-735c-4582-b54f-f63c0e0663fb-kube-api-access-n7lhz\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.027509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.026379    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5vfp\" (UniqueName: \"kubernetes.io/projected/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-kube-api-access-j5vfp\") pod \"network-metrics-daemon-8fkfq\" (UID: \"bfd52bcf-e224-4a4c-8fa2-13c94f985e60\") " pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:19:49.027509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.026409    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:49.027509 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.026538    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:19:49.027509 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.026603    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret podName:46b91743-4ee2-4ac1-9779-5b85586520ec nodeName:}" failed. No retries permitted until 2026-08-05 17:19:49.526576487 +0000 UTC m=+2.944891226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret") pod "global-pull-secret-syncer-t2vbt" (UID: "46b91743-4ee2-4ac1-9779-5b85586520ec") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:19:49.027509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.026861    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6489ed96-823a-4ff5-bab4-0efabcb3584b-konnectivity-ca\") pod \"konnectivity-agent-q4cgr\" (UID: \"6489ed96-823a-4ff5-bab4-0efabcb3584b\") " pod="kube-system/konnectivity-agent-q4cgr"
Aug 05 17:19:49.030761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.030740    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6489ed96-823a-4ff5-bab4-0efabcb3584b-agent-certs\") pod \"konnectivity-agent-q4cgr\" (UID: \"6489ed96-823a-4ff5-bab4-0efabcb3584b\") " pod="kube-system/konnectivity-agent-q4cgr"
Aug 05 17:19:49.032727 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.032567    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-tmp\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.034287 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.034267    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jls2l\" (UniqueName: \"kubernetes.io/projected/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-kube-api-access-jls2l\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.034478 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.034458    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/55eb403f-651a-4ce8-a74b-ecb40eb0f16d-etc-tuned\") pod \"tuned-pgfr7\" (UID: \"55eb403f-651a-4ce8-a74b-ecb40eb0f16d\") " pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.042839 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.042817    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh96h\" (UniqueName: \"kubernetes.io/projected/e8d949ff-bbbc-4247-87f0-d9f98e7ec389-kube-api-access-nh96h\") pod \"node-resolver-5z6td\" (UID: \"e8d949ff-bbbc-4247-87f0-d9f98e7ec389\") " pod="openshift-dns/node-resolver-5z6td"
Aug 05 17:19:49.043775 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.043138    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rspm\" (UniqueName: \"kubernetes.io/projected/c2873311-84df-4d01-ac08-3deb1f12420c-kube-api-access-5rspm\") pod \"iptables-alerter-k2tgt\" (UID: \"c2873311-84df-4d01-ac08-3deb1f12420c\") " pod="openshift-network-operator/iptables-alerter-k2tgt"
Aug 05 17:19:49.126716 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.126681    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-device-dir\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.126873 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.126726    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqdc7\" (UniqueName: \"kubernetes.io/projected/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-kube-api-access-jqdc7\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.126873 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.126796    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-device-dir\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.126873 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.126850    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4d033317-735c-4582-b54f-f63c0e0663fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.127053 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.126890    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a264b228-b7f6-4977-b858-820fc604b53c-serviceca\") pod \"node-ca-l2prc\" (UID: \"a264b228-b7f6-4977-b858-820fc604b53c\") " pod="openshift-image-registry/node-ca-l2prc"
Aug 05 17:19:49.127053 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.126916    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5qjt\" (UniqueName: \"kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt\") pod \"network-check-target-5zfsf\" (UID: \"e9766cbc-f4f7-402d-9f9b-15e132d49b71\") " pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:19:49.127053 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.126942    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-cni-bin\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.127053 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.126968    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpn46\" (UniqueName: \"kubernetes.io/projected/658875a2-a766-417a-b50e-ee46f5ba179b-kube-api-access-bpn46\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127359    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4d033317-735c-4582-b54f-f63c0e0663fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127395    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs\") pod \"network-metrics-daemon-8fkfq\" (UID: \"bfd52bcf-e224-4a4c-8fa2-13c94f985e60\") " pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127421    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/658875a2-a766-417a-b50e-ee46f5ba179b-env-overrides\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127361    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-cni-bin\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127444    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127468    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj8bt\" (UniqueName: \"kubernetes.io/projected/a264b228-b7f6-4977-b858-820fc604b53c-kube-api-access-fj8bt\") pod \"node-ca-l2prc\" (UID: \"a264b228-b7f6-4977-b858-820fc604b53c\") " pod="openshift-image-registry/node-ca-l2prc"
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127494    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-cnibin\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127511    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4d033317-735c-4582-b54f-f63c0e0663fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127517    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-multus-socket-dir-parent\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.127535    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127540    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-run-ovn\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127549    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4d033317-735c-4582-b54f-f63c0e0663fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127563    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/658875a2-a766-417a-b50e-ee46f5ba179b-ovnkube-script-lib\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127584    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-cni-binary-copy\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.127599    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs podName:bfd52bcf-e224-4a4c-8fa2-13c94f985e60 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:49.627581165 +0000 UTC m=+3.045895904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs") pod "network-metrics-daemon-8fkfq" (UID: "bfd52bcf-e224-4a4c-8fa2-13c94f985e60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127625    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-run-k8s-cni-cncf-io\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.128565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127652    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-var-lib-cni-bin\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127669    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-multus-daemon-config\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127683    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-run-netns\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127698    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-etc-kubernetes\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127715    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-run-netns\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127734    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a264b228-b7f6-4977-b858-820fc604b53c-serviceca\") pod \"node-ca-l2prc\" (UID: \"a264b228-b7f6-4977-b858-820fc604b53c\") " pod="openshift-image-registry/node-ca-l2prc"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127739    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127767    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127789    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-system-cni-dir\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127817    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4d033317-735c-4582-b54f-f63c0e0663fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127828    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127846    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d033317-735c-4582-b54f-f63c0e0663fb-system-cni-dir\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127872    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4d033317-735c-4582-b54f-f63c0e0663fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127894    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-multus-socket-dir-parent\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127940    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-var-lib-cni-bin\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127956    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-cnibin\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127898    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7lhz\" (UniqueName: \"kubernetes.io/projected/4d033317-735c-4582-b54f-f63c0e0663fb-kube-api-access-n7lhz\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.129520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127995    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-system-cni-dir\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.127996    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5vfp\" (UniqueName: \"kubernetes.io/projected/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-kube-api-access-j5vfp\") pod \"network-metrics-daemon-8fkfq\" (UID: \"bfd52bcf-e224-4a4c-8fa2-13c94f985e60\") " pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128000    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/658875a2-a766-417a-b50e-ee46f5ba179b-env-overrides\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128046    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-multus-cni-dir\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128071    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4d033317-735c-4582-b54f-f63c0e0663fb-cnibin\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128096    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4d033317-735c-4582-b54f-f63c0e0663fb-os-release\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128101    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-cni-binary-copy\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128119    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-log-socket\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128142    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/658875a2-a766-417a-b50e-ee46f5ba179b-ovn-node-metrics-cert\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128196    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-registration-dir\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128226    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zgdp\" (UniqueName: \"kubernetes.io/projected/29967c90-6d3d-43b1-a472-714f10ab92ab-kube-api-access-4zgdp\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128250    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-slash\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128269    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4d033317-735c-4582-b54f-f63c0e0663fb-system-cni-dir\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128282    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128307    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-socket-dir\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128330    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-var-lib-openvswitch\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128342    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-run-netns\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.130121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128354    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-node-log\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128383    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-run-multus-certs\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128414    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-multus-cni-dir\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128353    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4d033317-735c-4582-b54f-f63c0e0663fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128228    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-run-k8s-cni-cncf-io\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128415    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-run-ovn-kubernetes\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128458    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-socket-dir\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128467    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-cni-netd\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128309    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-etc-kubernetes\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128475    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4d033317-735c-4582-b54f-f63c0e0663fb-cnibin\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128479    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-var-lib-openvswitch\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128346    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-run-netns\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128440    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-node-log\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128514    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-cni-netd\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128517    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128519    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-multus-daemon-config\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128535    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-run-multus-certs\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.130959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128543    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-var-lib-cni-multus\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128559    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-run-ovn-kubernetes\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128567    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-slash\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128570    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-run-systemd\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128597    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-run-systemd\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128605    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/658875a2-a766-417a-b50e-ee46f5ba179b-ovnkube-script-lib\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128608    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/658875a2-a766-417a-b50e-ee46f5ba179b-ovnkube-config\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128643    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-registration-dir\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128653    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-log-socket\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128655    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-sys-fs\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128680    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-var-lib-cni-multus\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128696    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/29967c90-6d3d-43b1-a472-714f10ab92ab-sys-fs\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128698    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-var-lib-kubelet\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128711    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-run-ovn\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128724    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-hostroot\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128725    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-host-var-lib-kubelet\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128751    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-multus-conf-dir\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128778    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-kubelet\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.131759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128783    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-hostroot\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128786    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4d033317-735c-4582-b54f-f63c0e0663fb-os-release\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128805    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-etc-openvswitch\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128807    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-multus-conf-dir\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128830    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a264b228-b7f6-4977-b858-820fc604b53c-host\") pod \"node-ca-l2prc\" (UID: \"a264b228-b7f6-4977-b858-820fc604b53c\") " pod="openshift-image-registry/node-ca-l2prc"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128865    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-etc-openvswitch\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128876    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a264b228-b7f6-4977-b858-820fc604b53c-host\") pod \"node-ca-l2prc\" (UID: \"a264b228-b7f6-4977-b858-820fc604b53c\") " pod="openshift-image-registry/node-ca-l2prc"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128871    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-host-kubelet\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128893    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-os-release\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128919    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-systemd-units\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128935    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-os-release\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128945    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-run-openvswitch\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128959    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-systemd-units\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128978    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/658875a2-a766-417a-b50e-ee46f5ba179b-run-openvswitch\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.128999    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4d033317-735c-4582-b54f-f63c0e0663fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.129064    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/658875a2-a766-417a-b50e-ee46f5ba179b-ovnkube-config\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.132669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.131329    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/658875a2-a766-417a-b50e-ee46f5ba179b-ovn-node-metrics-cert\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.135343 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.135323    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:19:49.135444 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.135348    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:19:49.135444 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.135360    2556 projected.go:196] Error preparing data for projected volume kube-api-access-j5qjt for pod openshift-network-diagnostics/network-check-target-5zfsf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:19:49.135444 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.135416    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt podName:e9766cbc-f4f7-402d-9f9b-15e132d49b71 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:49.635403917 +0000 UTC m=+3.053718651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-j5qjt" (UniqueName: "kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt") pod "network-check-target-5zfsf" (UID: "e9766cbc-f4f7-402d-9f9b-15e132d49b71") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:19:49.138137 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.138063    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5vfp\" (UniqueName: \"kubernetes.io/projected/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-kube-api-access-j5vfp\") pod \"network-metrics-daemon-8fkfq\" (UID: \"bfd52bcf-e224-4a4c-8fa2-13c94f985e60\") " pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:19:49.138669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.138647    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj8bt\" (UniqueName: \"kubernetes.io/projected/a264b228-b7f6-4977-b858-820fc604b53c-kube-api-access-fj8bt\") pod \"node-ca-l2prc\" (UID: \"a264b228-b7f6-4977-b858-820fc604b53c\") " pod="openshift-image-registry/node-ca-l2prc"
Aug 05 17:19:49.139250 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.139212    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqdc7\" (UniqueName: \"kubernetes.io/projected/acc396cc-8191-48c6-8ba3-c6e0fd95f0fa-kube-api-access-jqdc7\") pod \"multus-74dt8\" (UID: \"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa\") " pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.139977 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.139948    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpn46\" (UniqueName: \"kubernetes.io/projected/658875a2-a766-417a-b50e-ee46f5ba179b-kube-api-access-bpn46\") pod \"ovnkube-node-l9ptq\" (UID: \"658875a2-a766-417a-b50e-ee46f5ba179b\") " pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.140485 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.140380    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zgdp\" (UniqueName: \"kubernetes.io/projected/29967c90-6d3d-43b1-a472-714f10ab92ab-kube-api-access-4zgdp\") pod \"aws-ebs-csi-driver-node-c48t7\" (UID: \"29967c90-6d3d-43b1-a472-714f10ab92ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.141210 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.141122    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7lhz\" (UniqueName: \"kubernetes.io/projected/4d033317-735c-4582-b54f-f63c0e0663fb-kube-api-access-n7lhz\") pod \"multus-additional-cni-plugins-rzlln\" (UID: \"4d033317-735c-4582-b54f-f63c0e0663fb\") " pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.191508 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.191454    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:19:49.217819 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.217798    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5z6td"
Aug 05 17:19:49.229834 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.229817    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pgfr7"
Aug 05 17:19:49.240459 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.240441    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k2tgt"
Aug 05 17:19:49.245079 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.245057    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q4cgr"
Aug 05 17:19:49.254146 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.254125    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-l2prc"
Aug 05 17:19:49.263526 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.263507    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-74dt8"
Aug 05 17:19:49.278977 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.278959    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzlln"
Aug 05 17:19:49.289487 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.289472    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:19:49.294004 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.293986    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7"
Aug 05 17:19:49.531517 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.531442    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:49.531669 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.531551    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:19:49.531669 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.531614    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret podName:46b91743-4ee2-4ac1-9779-5b85586520ec nodeName:}" failed. No retries permitted until 2026-08-05 17:19:50.531589071 +0000 UTC m=+3.949903807 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret") pod "global-pull-secret-syncer-t2vbt" (UID: "46b91743-4ee2-4ac1-9779-5b85586520ec") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:19:49.631758 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.631732    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs\") pod \"network-metrics-daemon-8fkfq\" (UID: \"bfd52bcf-e224-4a4c-8fa2-13c94f985e60\") " pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:19:49.631920 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.631839    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:19:49.631920 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.631892    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs podName:bfd52bcf-e224-4a4c-8fa2-13c94f985e60 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:50.631876498 +0000 UTC m=+4.050191234 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs") pod "network-metrics-daemon-8fkfq" (UID: "bfd52bcf-e224-4a4c-8fa2-13c94f985e60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:19:49.733004 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.732975    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5qjt\" (UniqueName: \"kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt\") pod \"network-check-target-5zfsf\" (UID: \"e9766cbc-f4f7-402d-9f9b-15e132d49b71\") " pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:19:49.733163 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.733115    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:19:49.733163 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.733131    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:19:49.733163 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.733139    2556 projected.go:196] Error preparing data for projected volume kube-api-access-j5qjt for pod openshift-network-diagnostics/network-check-target-5zfsf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:19:49.733266 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:49.733196    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt podName:e9766cbc-f4f7-402d-9f9b-15e132d49b71 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:50.733182793 +0000 UTC m=+4.151497528 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-j5qjt" (UniqueName: "kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt") pod "network-check-target-5zfsf" (UID: "e9766cbc-f4f7-402d-9f9b-15e132d49b71") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:19:49.956289 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.956209    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-04 17:14:48 +0000 UTC" deadline="2028-05-08 18:01:00.19829671 +0000 UTC"
Aug 05 17:19:49.956289 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:49.956244    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15408h41m10.242060455s"
Aug 05 17:19:50.042005 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:50.041980    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2873311_84df_4d01_ac08_3deb1f12420c.slice/crio-1386b67f9ef7ac4a514c6746ff888b80414b779edc1f8f27df69cd109b9a9f06 WatchSource:0}: Error finding container 1386b67f9ef7ac4a514c6746ff888b80414b779edc1f8f27df69cd109b9a9f06: Status 404 returned error can't find the container with id 1386b67f9ef7ac4a514c6746ff888b80414b779edc1f8f27df69cd109b9a9f06
Aug 05 17:19:50.066803 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:50.066780    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29967c90_6d3d_43b1_a472_714f10ab92ab.slice/crio-8031eb91c5960369cab759c08615d996c2b9447be6273ff55e8ba384f01cb802 WatchSource:0}: Error finding container 8031eb91c5960369cab759c08615d996c2b9447be6273ff55e8ba384f01cb802: Status 404 returned error can't find the container with id 8031eb91c5960369cab759c08615d996c2b9447be6273ff55e8ba384f01cb802
Aug 05 17:19:50.067812 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:50.067787    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacc396cc_8191_48c6_8ba3_c6e0fd95f0fa.slice/crio-dd20176e5346bd13618807a0495916d362ab8d800e64d5681c8a06b767daeaa8 WatchSource:0}: Error finding container dd20176e5346bd13618807a0495916d362ab8d800e64d5681c8a06b767daeaa8: Status 404 returned error can't find the container with id dd20176e5346bd13618807a0495916d362ab8d800e64d5681c8a06b767daeaa8
Aug 05 17:19:50.068587 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:50.068566    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda264b228_b7f6_4977_b858_820fc604b53c.slice/crio-a024a5c3c6f962e2e75147b9c18385e4fb361787888a5e1ef5cf037a71ef90d2 WatchSource:0}: Error finding container a024a5c3c6f962e2e75147b9c18385e4fb361787888a5e1ef5cf037a71ef90d2: Status 404 returned error can't find the container with id a024a5c3c6f962e2e75147b9c18385e4fb361787888a5e1ef5cf037a71ef90d2
Aug 05 17:19:50.070133 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:50.070002    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8d949ff_bbbc_4247_87f0_d9f98e7ec389.slice/crio-e2d839c79f0cf511e7b190c826fcf6dfe3e49ef2dc15535345f33a2b64dcbb94 WatchSource:0}: Error finding container e2d839c79f0cf511e7b190c826fcf6dfe3e49ef2dc15535345f33a2b64dcbb94: Status 404 returned error can't find the container with id e2d839c79f0cf511e7b190c826fcf6dfe3e49ef2dc15535345f33a2b64dcbb94
Aug 05 17:19:50.071446 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:50.071421    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d033317_735c_4582_b54f_f63c0e0663fb.slice/crio-f420f6dae2e2409f551591989ce43f3a75ce3305db4a24fd378b313628089766 WatchSource:0}: Error finding container f420f6dae2e2409f551591989ce43f3a75ce3305db4a24fd378b313628089766: Status 404 returned error can't find the container with id f420f6dae2e2409f551591989ce43f3a75ce3305db4a24fd378b313628089766
Aug 05 17:19:50.072332 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:50.072308    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55eb403f_651a_4ce8_a74b_ecb40eb0f16d.slice/crio-c1229ae0843a4c1e5550e2dba383794859013a1b7d9c3ad1b8e697f64a459268 WatchSource:0}: Error finding container c1229ae0843a4c1e5550e2dba383794859013a1b7d9c3ad1b8e697f64a459268: Status 404 returned error can't find the container with id c1229ae0843a4c1e5550e2dba383794859013a1b7d9c3ad1b8e697f64a459268
Aug 05 17:19:50.075924 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:19:50.075824    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod658875a2_a766_417a_b50e_ee46f5ba179b.slice/crio-a6832ca5a9f02cc3e212d38c5bc6131bf1c1dc96d2ee37edfe6ec096bbe3207b WatchSource:0}: Error finding container a6832ca5a9f02cc3e212d38c5bc6131bf1c1dc96d2ee37edfe6ec096bbe3207b: Status 404 returned error can't find the container with id a6832ca5a9f02cc3e212d38c5bc6131bf1c1dc96d2ee37edfe6ec096bbe3207b
Aug 05 17:19:50.101054 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:50.101024    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l2prc" event={"ID":"a264b228-b7f6-4977-b858-820fc604b53c","Type":"ContainerStarted","Data":"a024a5c3c6f962e2e75147b9c18385e4fb361787888a5e1ef5cf037a71ef90d2"}
Aug 05 17:19:50.103317 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:50.103293    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq" event={"ID":"658875a2-a766-417a-b50e-ee46f5ba179b","Type":"ContainerStarted","Data":"a6832ca5a9f02cc3e212d38c5bc6131bf1c1dc96d2ee37edfe6ec096bbe3207b"}
Aug 05 17:19:50.107712 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:50.107689    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pgfr7" event={"ID":"55eb403f-651a-4ce8-a74b-ecb40eb0f16d","Type":"ContainerStarted","Data":"c1229ae0843a4c1e5550e2dba383794859013a1b7d9c3ad1b8e697f64a459268"}
Aug 05 17:19:50.110594 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:50.110572    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzlln" event={"ID":"4d033317-735c-4582-b54f-f63c0e0663fb","Type":"ContainerStarted","Data":"f420f6dae2e2409f551591989ce43f3a75ce3305db4a24fd378b313628089766"}
Aug 05 17:19:50.111521 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:50.111502    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-74dt8" event={"ID":"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa","Type":"ContainerStarted","Data":"dd20176e5346bd13618807a0495916d362ab8d800e64d5681c8a06b767daeaa8"}
Aug 05 17:19:50.112427 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:50.112409    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7" event={"ID":"29967c90-6d3d-43b1-a472-714f10ab92ab","Type":"ContainerStarted","Data":"8031eb91c5960369cab759c08615d996c2b9447be6273ff55e8ba384f01cb802"}
Aug 05 17:19:50.113732 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:50.113457    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k2tgt" event={"ID":"c2873311-84df-4d01-ac08-3deb1f12420c","Type":"ContainerStarted","Data":"1386b67f9ef7ac4a514c6746ff888b80414b779edc1f8f27df69cd109b9a9f06"}
Aug 05 17:19:50.114561 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:50.114542    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q4cgr" event={"ID":"6489ed96-823a-4ff5-bab4-0efabcb3584b","Type":"ContainerStarted","Data":"c97c8e7a45ae5da6fbf45db031af941cb05399ef23b38479dca7a083ee5f4f77"}
Aug 05 17:19:50.115495 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:50.115470    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5z6td" event={"ID":"e8d949ff-bbbc-4247-87f0-d9f98e7ec389","Type":"ContainerStarted","Data":"e2d839c79f0cf511e7b190c826fcf6dfe3e49ef2dc15535345f33a2b64dcbb94"}
Aug 05 17:19:50.538704 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:50.538364    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:50.538859 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:50.538758    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:19:50.538859 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:50.538827    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret podName:46b91743-4ee2-4ac1-9779-5b85586520ec nodeName:}" failed. No retries permitted until 2026-08-05 17:19:52.538808387 +0000 UTC m=+5.957123125 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret") pod "global-pull-secret-syncer-t2vbt" (UID: "46b91743-4ee2-4ac1-9779-5b85586520ec") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:19:50.639034 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:50.638997    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs\") pod \"network-metrics-daemon-8fkfq\" (UID: \"bfd52bcf-e224-4a4c-8fa2-13c94f985e60\") " pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:19:50.639229 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:50.639142    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:19:50.639229 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:50.639225    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs podName:bfd52bcf-e224-4a4c-8fa2-13c94f985e60 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:52.639204557 +0000 UTC m=+6.057519294 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs") pod "network-metrics-daemon-8fkfq" (UID: "bfd52bcf-e224-4a4c-8fa2-13c94f985e60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:19:50.739367 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:50.739329    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5qjt\" (UniqueName: \"kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt\") pod \"network-check-target-5zfsf\" (UID: \"e9766cbc-f4f7-402d-9f9b-15e132d49b71\") " pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:19:50.739531 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:50.739512    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:19:50.739611 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:50.739536    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:19:50.739611 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:50.739549    2556 projected.go:196] Error preparing data for projected volume kube-api-access-j5qjt for pod openshift-network-diagnostics/network-check-target-5zfsf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:19:50.739719 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:50.739615    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt podName:e9766cbc-f4f7-402d-9f9b-15e132d49b71 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:52.739597506 +0000 UTC m=+6.157912241 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-j5qjt" (UniqueName: "kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt") pod "network-check-target-5zfsf" (UID: "e9766cbc-f4f7-402d-9f9b-15e132d49b71") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:19:50.956982 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:50.956871    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-04 17:14:48 +0000 UTC" deadline="2028-02-08 20:19:27.458490748 +0000 UTC"
Aug 05 17:19:50.956982 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:50.956928    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13250h59m36.501566793s"
Aug 05 17:19:51.091239 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:51.091194    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:19:51.091239 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:51.091217    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:51.091461 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:51.091194    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:19:51.091461 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:51.091335    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:19:51.091572 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:51.091454    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:19:51.091572 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:51.091536    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:19:51.125172 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:51.124794    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal" event={"ID":"13b14f86a6dd04037249bf3e254a8bd2","Type":"ContainerStarted","Data":"751bee254ce38617739d7e774a287026becf4d9dcd712e9775c0efeb8a2024e9"}
Aug 05 17:19:51.144530 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:51.144477    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-37.ec2.internal" podStartSLOduration=3.144458508 podStartE2EDuration="3.144458508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:19:51.143903403 +0000 UTC m=+4.562218162" watchObservedRunningTime="2026-08-05 17:19:51.144458508 +0000 UTC m=+4.562773267"
Aug 05 17:19:51.148854 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:51.148168    2556 generic.go:356] "Generic (PLEG): container finished" podID="29384c2fdf1ce8bc484da2b88a670c44" containerID="f1b3dc2e9cb91aae250e394f7362d22fba7615323f0596f455cdfbb9fc30c995" exitCode=0
Aug 05 17:19:51.148854 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:51.148209    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" event={"ID":"29384c2fdf1ce8bc484da2b88a670c44","Type":"ContainerDied","Data":"f1b3dc2e9cb91aae250e394f7362d22fba7615323f0596f455cdfbb9fc30c995"}
Aug 05 17:19:52.173230 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:52.173191    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" event={"ID":"29384c2fdf1ce8bc484da2b88a670c44","Type":"ContainerStarted","Data":"2cc1ffb8ec12b1c5a6645f5f4b2e87392ff48762b35e131f5b232bce617d90eb"}
Aug 05 17:19:52.555254 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:52.555173    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:52.555418 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:52.555388    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:19:52.555481 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:52.555456    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret podName:46b91743-4ee2-4ac1-9779-5b85586520ec nodeName:}" failed. No retries permitted until 2026-08-05 17:19:56.555436273 +0000 UTC m=+9.973751023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret") pod "global-pull-secret-syncer-t2vbt" (UID: "46b91743-4ee2-4ac1-9779-5b85586520ec") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:19:52.656013 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:52.655956    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs\") pod \"network-metrics-daemon-8fkfq\" (UID: \"bfd52bcf-e224-4a4c-8fa2-13c94f985e60\") " pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:19:52.656221 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:52.656095    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:19:52.656221 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:52.656176    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs podName:bfd52bcf-e224-4a4c-8fa2-13c94f985e60 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:56.656144015 +0000 UTC m=+10.074458752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs") pod "network-metrics-daemon-8fkfq" (UID: "bfd52bcf-e224-4a4c-8fa2-13c94f985e60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:19:52.757057 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:52.757020    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5qjt\" (UniqueName: \"kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt\") pod \"network-check-target-5zfsf\" (UID: \"e9766cbc-f4f7-402d-9f9b-15e132d49b71\") " pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:19:52.757258 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:52.757235    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:19:52.757258 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:52.757256    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:19:52.757380 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:52.757270    2556 projected.go:196] Error preparing data for projected volume kube-api-access-j5qjt for pod openshift-network-diagnostics/network-check-target-5zfsf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:19:52.757380 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:52.757327    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt podName:e9766cbc-f4f7-402d-9f9b-15e132d49b71 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:56.757307622 +0000 UTC m=+10.175622361 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-j5qjt" (UniqueName: "kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt") pod "network-check-target-5zfsf" (UID: "e9766cbc-f4f7-402d-9f9b-15e132d49b71") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:19:53.093702 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:53.092933    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:19:53.093702 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:53.093075    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:19:53.093702 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:53.093453    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:19:53.093702 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:53.093542    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:19:53.093702 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:53.093592    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:53.093702 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:53.093670    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:19:55.090944 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:55.090913    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:55.091366 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:55.091045    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:19:55.091571 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:55.091542    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:19:55.091694 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:55.091672    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:19:55.091882 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:55.091851    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:19:55.092019 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:55.091937    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:19:56.588582 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:56.588547    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:56.589017 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:56.588724    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:19:56.589017 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:56.588784    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret podName:46b91743-4ee2-4ac1-9779-5b85586520ec nodeName:}" failed. No retries permitted until 2026-08-05 17:20:04.588766074 +0000 UTC m=+18.007080823 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret") pod "global-pull-secret-syncer-t2vbt" (UID: "46b91743-4ee2-4ac1-9779-5b85586520ec") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:19:56.689352 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:56.689247    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs\") pod \"network-metrics-daemon-8fkfq\" (UID: \"bfd52bcf-e224-4a4c-8fa2-13c94f985e60\") " pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:19:56.689529 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:56.689389    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:19:56.689529 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:56.689453    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs podName:bfd52bcf-e224-4a4c-8fa2-13c94f985e60 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:04.689435487 +0000 UTC m=+18.107750230 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs") pod "network-metrics-daemon-8fkfq" (UID: "bfd52bcf-e224-4a4c-8fa2-13c94f985e60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:19:56.790055 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:56.790020    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5qjt\" (UniqueName: \"kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt\") pod \"network-check-target-5zfsf\" (UID: \"e9766cbc-f4f7-402d-9f9b-15e132d49b71\") " pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:19:56.790232 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:56.790203    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:19:56.790232 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:56.790222    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:19:56.790232 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:56.790234    2556 projected.go:196] Error preparing data for projected volume kube-api-access-j5qjt for pod openshift-network-diagnostics/network-check-target-5zfsf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:19:56.790382 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:56.790287    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt podName:e9766cbc-f4f7-402d-9f9b-15e132d49b71 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:04.790273041 +0000 UTC m=+18.208587775 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-j5qjt" (UniqueName: "kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt") pod "network-check-target-5zfsf" (UID: "e9766cbc-f4f7-402d-9f9b-15e132d49b71") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:19:57.092176 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:57.091819    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:57.092176 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:57.091929    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:19:57.092176 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:57.092129    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:19:57.092447 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:57.092248    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:19:57.092447 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:57.092296    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:19:57.092447 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:57.092350    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:19:59.091449 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:59.091416    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:19:59.091926 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:59.091427    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:19:59.091926 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:59.091568    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:19:59.091926 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:19:59.091429    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:19:59.091926 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:59.091641    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:19:59.091926 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:19:59.091741    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:20:01.091008 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:01.090980    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:01.091008 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:01.090999    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:01.091536 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:01.090980    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:01.091536 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:01.091105    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:20:01.091536 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:01.091200    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:20:01.091536 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:01.091271    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:20:03.090748 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:03.090716    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:03.091247 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:03.090718    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:03.091247 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:03.090829    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:20:03.091247 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:03.090715    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:03.091247 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:03.090904    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:20:03.091247 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:03.090974    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:20:04.648956 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:04.648919    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:04.649442 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:04.649086    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:20:04.649442 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:04.649174    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret podName:46b91743-4ee2-4ac1-9779-5b85586520ec nodeName:}" failed. No retries permitted until 2026-08-05 17:20:20.649139218 +0000 UTC m=+34.067453958 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret") pod "global-pull-secret-syncer-t2vbt" (UID: "46b91743-4ee2-4ac1-9779-5b85586520ec") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:20:04.749941 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:04.749898    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs\") pod \"network-metrics-daemon-8fkfq\" (UID: \"bfd52bcf-e224-4a4c-8fa2-13c94f985e60\") " pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:04.750096 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:04.750014    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:20:04.750096 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:04.750069    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs podName:bfd52bcf-e224-4a4c-8fa2-13c94f985e60 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:20.750051515 +0000 UTC m=+34.168366254 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs") pod "network-metrics-daemon-8fkfq" (UID: "bfd52bcf-e224-4a4c-8fa2-13c94f985e60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:20:04.850417 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:04.850383    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5qjt\" (UniqueName: \"kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt\") pod \"network-check-target-5zfsf\" (UID: \"e9766cbc-f4f7-402d-9f9b-15e132d49b71\") " pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:04.850591 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:04.850560    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:20:04.850591 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:04.850582    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:20:04.850591 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:04.850592    2556 projected.go:196] Error preparing data for projected volume kube-api-access-j5qjt for pod openshift-network-diagnostics/network-check-target-5zfsf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:20:04.850742 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:04.850651    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt podName:e9766cbc-f4f7-402d-9f9b-15e132d49b71 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:20.850633218 +0000 UTC m=+34.268947973 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-j5qjt" (UniqueName: "kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt") pod "network-check-target-5zfsf" (UID: "e9766cbc-f4f7-402d-9f9b-15e132d49b71") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:20:05.090754 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:05.090720    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:05.090938 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:05.090778    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:05.090938 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:05.090728    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:05.090938 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:05.090843    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:20:05.091070 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:05.090939    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:20:05.091166 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:05.091112    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:20:07.091261 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:07.091221    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:07.091723 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:07.091305    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:07.091723 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:07.091326    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:20:07.091723 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:07.091403    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:20:07.091723 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:07.091506    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:07.091723 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:07.091585    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:20:09.092086 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.091711    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:09.092904 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.091711    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:09.092904 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:09.092215    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:20:09.092904 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:09.092277    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:20:09.092904 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.091725    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:09.092904 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:09.092346    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:20:09.204583 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.204554    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq" event={"ID":"658875a2-a766-417a-b50e-ee46f5ba179b","Type":"ContainerStarted","Data":"d7e067da8c0d9579f2e750ac08e495ba1b08cfbce98abdcd3d42170848d3759b"}
Aug 05 17:20:09.204674 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.204622    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq" event={"ID":"658875a2-a766-417a-b50e-ee46f5ba179b","Type":"ContainerStarted","Data":"bebaa2602b7e9eb1e8251291743b9aa06109c082a8065ccf1221574cc41d83c6"}
Aug 05 17:20:09.204674 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.204660    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq" event={"ID":"658875a2-a766-417a-b50e-ee46f5ba179b","Type":"ContainerStarted","Data":"9aae82397c2016fbbdc7e11d167b22a8c1dc27cc9b3b2137ca0b2322644596c2"}
Aug 05 17:20:09.204766 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.204693    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq" event={"ID":"658875a2-a766-417a-b50e-ee46f5ba179b","Type":"ContainerStarted","Data":"95ba05d55c3dfd7feece5720adce1313e89039d6ce16489b62a6e7ffb9db0b7a"}
Aug 05 17:20:09.204766 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.204726    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq" event={"ID":"658875a2-a766-417a-b50e-ee46f5ba179b","Type":"ContainerStarted","Data":"143cd4c6dc5e7107a7713fb2048c0750c69d8a249564c8e87bd84fe4c53ab0f8"}
Aug 05 17:20:09.205892 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.205866    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pgfr7" event={"ID":"55eb403f-651a-4ce8-a74b-ecb40eb0f16d","Type":"ContainerStarted","Data":"a89900e62fabd1c63ee9cd5b9c54b07de0d9a26f6d558a54f1756352d5ad6fc5"}
Aug 05 17:20:09.207257 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.207232    2556 generic.go:356] "Generic (PLEG): container finished" podID="4d033317-735c-4582-b54f-f63c0e0663fb" containerID="c5030d7f8022ebeb8f8264609f14cc7089e4732b9116b94e44bba5f410c47452" exitCode=0
Aug 05 17:20:09.207359 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.207315    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzlln" event={"ID":"4d033317-735c-4582-b54f-f63c0e0663fb","Type":"ContainerDied","Data":"c5030d7f8022ebeb8f8264609f14cc7089e4732b9116b94e44bba5f410c47452"}
Aug 05 17:20:09.208761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.208739    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-74dt8" event={"ID":"acc396cc-8191-48c6-8ba3-c6e0fd95f0fa","Type":"ContainerStarted","Data":"8e5cfc9605e4974b2cb95aa393cf4d76695a6f0a0101cfefbc20590a99aa335b"}
Aug 05 17:20:09.210268 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.210234    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7" event={"ID":"29967c90-6d3d-43b1-a472-714f10ab92ab","Type":"ContainerStarted","Data":"954e935c75901ff7fda263f8541eb5214822116e9c4cb3d467d3814a0f56e084"}
Aug 05 17:20:09.211690 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.211656    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q4cgr" event={"ID":"6489ed96-823a-4ff5-bab4-0efabcb3584b","Type":"ContainerStarted","Data":"d34a7d906e831b49ad5ef32799aba811329c3798bcb6c0cfc14d1f57b8eead6d"}
Aug 05 17:20:09.212877 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.212858    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5z6td" event={"ID":"e8d949ff-bbbc-4247-87f0-d9f98e7ec389","Type":"ContainerStarted","Data":"5dc70532f79c0470854349045e46fc48a87236966fb406fcdc5cc5257eae99da"}
Aug 05 17:20:09.214180 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.214139    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-l2prc" event={"ID":"a264b228-b7f6-4977-b858-820fc604b53c","Type":"ContainerStarted","Data":"2ff29d457999e5ea4a393e7c1b6d8bbf3d4c27a234f7bd93b1516143fb9f2dca"}
Aug 05 17:20:09.244926 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.244888    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-37.ec2.internal" podStartSLOduration=21.244876971 podStartE2EDuration="21.244876971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:19:52.191543421 +0000 UTC m=+5.609858196" watchObservedRunningTime="2026-08-05 17:20:09.244876971 +0000 UTC m=+22.663191728"
Aug 05 17:20:09.311182 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.311102    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5z6td" podStartSLOduration=4.002656949 podStartE2EDuration="22.31108028s" totalImagesPullingTime="18.308423331s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:09.281481949 +0000 UTC m=+22.699796719" watchObservedRunningTime="2026-08-05 17:20:09.31108028 +0000 UTC m=+22.729395038"
Aug 05 17:20:09.311447 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.311410    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-74dt8" podStartSLOduration=3.710262304 podStartE2EDuration="22.311396523s" totalImagesPullingTime="18.601134219s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:09.311338438 +0000 UTC m=+22.729653188" watchObservedRunningTime="2026-08-05 17:20:09.311396523 +0000 UTC m=+22.729711281"
Aug 05 17:20:09.329766 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.329713    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pgfr7" podStartSLOduration=4.029787529 podStartE2EDuration="22.329696276s" totalImagesPullingTime="18.299908747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:09.32800007 +0000 UTC m=+22.746314827" watchObservedRunningTime="2026-08-05 17:20:09.329696276 +0000 UTC m=+22.748011034"
Aug 05 17:20:09.363881 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.363832    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-q4cgr" podStartSLOduration=4.065827359 podStartE2EDuration="22.363813027s" totalImagesPullingTime="18.297985668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:19:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:09.346117854 +0000 UTC m=+22.764432610" watchObservedRunningTime="2026-08-05 17:20:09.363813027 +0000 UTC m=+22.782127787"
Aug 05 17:20:09.364035 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.364009    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-l2prc" podStartSLOduration=4.115623049 podStartE2EDuration="22.363999666s" totalImagesPullingTime="18.248376617s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:09.363514455 +0000 UTC m=+22.781829233" watchObservedRunningTime="2026-08-05 17:20:09.363999666 +0000 UTC m=+22.782314426"
Aug 05 17:20:09.699906 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.699880    2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 05 17:20:09.979402 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.979265    2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-05T17:20:09.699896971Z","UUID":"91cdeb73-50e0-4202-8032-20286e111c12","Handler":null,"Name":"","Endpoint":""}
Aug 05 17:20:09.980993 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.980972    2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 05 17:20:09.981109 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:09.981013    2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 05 17:20:10.221129 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:10.221091    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq" event={"ID":"658875a2-a766-417a-b50e-ee46f5ba179b","Type":"ContainerStarted","Data":"0c7919b0483fae1691569546bd938c819aec211befccbe86b79f57a064f03c0b"}
Aug 05 17:20:10.223440 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:10.223408    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7" event={"ID":"29967c90-6d3d-43b1-a472-714f10ab92ab","Type":"ContainerStarted","Data":"0d3e75fe96ec1c8eb6db0092e95c8d1e9f716b71a554f3d449dc6be8fa3affc3"}
Aug 05 17:20:10.225941 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:10.225911    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k2tgt" event={"ID":"c2873311-84df-4d01-ac08-3deb1f12420c","Type":"ContainerStarted","Data":"227e38808934bec0002075485e01796e2aabd23c9d80ea562de1d1551bd254b3"}
Aug 05 17:20:10.241602 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:10.241522    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-k2tgt" podStartSLOduration=4.9267908590000005 podStartE2EDuration="23.241507388s" totalImagesPullingTime="18.314716529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:10.241393048 +0000 UTC m=+23.659707806" watchObservedRunningTime="2026-08-05 17:20:10.241507388 +0000 UTC m=+23.659822145"
Aug 05 17:20:11.091014 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:11.090979    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:11.091218 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:11.090981    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:11.091218 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:11.091086    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:20:11.091218 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:11.091113    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:11.091429 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:11.091229    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:20:11.091429 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:11.091310    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:20:11.230157 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:11.230125    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7" event={"ID":"29967c90-6d3d-43b1-a472-714f10ab92ab","Type":"ContainerStarted","Data":"73c6261a334a12611adbea25dc1fe597129302a6b02027d368798698f09d2990"}
Aug 05 17:20:11.249127 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:11.249078    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c48t7" podStartSLOduration=3.818670417 podStartE2EDuration="24.249061051s" totalImagesPullingTime="20.430390634s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:11.248554685 +0000 UTC m=+24.666869442" watchObservedRunningTime="2026-08-05 17:20:11.249061051 +0000 UTC m=+24.667375811"
Aug 05 17:20:12.237925 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:12.237876    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq" event={"ID":"658875a2-a766-417a-b50e-ee46f5ba179b","Type":"ContainerStarted","Data":"618d0aba6221b71dcba826b4329a6f052e76be87ae62f60bd6769b496453674e"}
Aug 05 17:20:13.091760 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:13.091505    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:13.092068 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:13.091872    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:20:13.092068 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:13.091510    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:13.092068 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:13.091509    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:13.092068 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:13.091981    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:20:13.092315 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:13.092067    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:20:13.653280 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:13.653253    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-q4cgr"
Aug 05 17:20:13.654116 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:13.654084    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-q4cgr"
Aug 05 17:20:14.245538 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:14.245244    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq" event={"ID":"658875a2-a766-417a-b50e-ee46f5ba179b","Type":"ContainerStarted","Data":"2e3585439d07949784de9f38d88b3073e86b37bfab4042f1c7b3381bca315314"}
Aug 05 17:20:14.245691 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:14.245554    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:20:14.245691 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:14.245615    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:20:14.245691 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:14.245646    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:20:14.247396 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:14.247371    2556 generic.go:356] "Generic (PLEG): container finished" podID="4d033317-735c-4582-b54f-f63c0e0663fb" containerID="05cf0986429dc978b93780d808b51952ed42bb07a241b720251a0995eaf3a421" exitCode=0
Aug 05 17:20:14.247522 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:14.247454    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzlln" event={"ID":"4d033317-735c-4582-b54f-f63c0e0663fb","Type":"ContainerDied","Data":"05cf0986429dc978b93780d808b51952ed42bb07a241b720251a0995eaf3a421"}
Aug 05 17:20:14.247601 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:14.247554    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-q4cgr"
Aug 05 17:20:14.248010 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:14.247993    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-q4cgr"
Aug 05 17:20:14.261861 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:14.261838    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:20:14.261946 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:14.261940    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:20:14.315044 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:14.315005    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq" podStartSLOduration=8.711845385 podStartE2EDuration="27.314992294s" totalImagesPullingTime="18.603146909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:14.278672176 +0000 UTC m=+27.696986934" watchObservedRunningTime="2026-08-05 17:20:14.314992294 +0000 UTC m=+27.733307050"
Aug 05 17:20:15.090813 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:15.090774    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:15.091353 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:15.090781    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:15.091353 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:15.090910    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:20:15.091353 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:15.090781    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:15.091353 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:15.090991    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:20:15.091353 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:15.091048    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:20:15.630532 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:15.630506    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8fkfq"]
Aug 05 17:20:15.630703 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:15.630608    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:15.630757 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:15.630706    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:20:15.633381 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:15.633352    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5zfsf"]
Aug 05 17:20:15.633506 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:15.633465    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:15.633570 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:15.633557    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:20:15.634078 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:15.634054    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t2vbt"]
Aug 05 17:20:15.634183 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:15.634135    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:15.634261 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:15.634241    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:20:16.253551 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:16.253518    2556 generic.go:356] "Generic (PLEG): container finished" podID="4d033317-735c-4582-b54f-f63c0e0663fb" containerID="f5f1faef5797af4dc2431fcb5bba2997957e396b2668c8dbec250309db42d956" exitCode=0
Aug 05 17:20:16.254004 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:16.253565    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzlln" event={"ID":"4d033317-735c-4582-b54f-f63c0e0663fb","Type":"ContainerDied","Data":"f5f1faef5797af4dc2431fcb5bba2997957e396b2668c8dbec250309db42d956"}
Aug 05 17:20:17.092245 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:17.092216    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:17.092446 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:17.092297    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:20:17.092446 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:17.092394    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:17.092536 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:17.092500    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:20:18.090913 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:18.090773    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:18.091356 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:18.091019    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:20:18.259649 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:18.259622    2556 generic.go:356] "Generic (PLEG): container finished" podID="4d033317-735c-4582-b54f-f63c0e0663fb" containerID="aaf92d64f31f04ef4b0a257209c5e44ecfc424eea752edadc61839473b49c36b" exitCode=0
Aug 05 17:20:18.259766 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:18.259697    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzlln" event={"ID":"4d033317-735c-4582-b54f-f63c0e0663fb","Type":"ContainerDied","Data":"aaf92d64f31f04ef4b0a257209c5e44ecfc424eea752edadc61839473b49c36b"}
Aug 05 17:20:19.090987 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:19.090952    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:19.091539 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:19.090968    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:19.091539 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:19.091099    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:20:19.091539 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:19.091183    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:20:20.091241 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:20.091206    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:20.091786 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:20.091322    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:20:20.670710 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:20.670675    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:20.670860 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:20.670793    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:20:20.670860 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:20.670853    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret podName:46b91743-4ee2-4ac1-9779-5b85586520ec nodeName:}" failed. No retries permitted until 2026-08-05 17:20:52.670837581 +0000 UTC m=+66.089152316 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret") pod "global-pull-secret-syncer-t2vbt" (UID: "46b91743-4ee2-4ac1-9779-5b85586520ec") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:20:20.771984 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:20.771950    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs\") pod \"network-metrics-daemon-8fkfq\" (UID: \"bfd52bcf-e224-4a4c-8fa2-13c94f985e60\") " pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:20.772146 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:20.772082    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:20:20.772224 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:20.772166    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs podName:bfd52bcf-e224-4a4c-8fa2-13c94f985e60 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:52.772136073 +0000 UTC m=+66.190450809 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs") pod "network-metrics-daemon-8fkfq" (UID: "bfd52bcf-e224-4a4c-8fa2-13c94f985e60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:20:20.873201 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:20.873140    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5qjt\" (UniqueName: \"kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt\") pod \"network-check-target-5zfsf\" (UID: \"e9766cbc-f4f7-402d-9f9b-15e132d49b71\") " pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:20.873363 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:20.873335    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:20:20.873363 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:20.873366    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:20:20.873541 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:20.873379    2556 projected.go:196] Error preparing data for projected volume kube-api-access-j5qjt for pod openshift-network-diagnostics/network-check-target-5zfsf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:20:20.873541 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:20.873440    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt podName:e9766cbc-f4f7-402d-9f9b-15e132d49b71 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:52.873421897 +0000 UTC m=+66.291736635 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-j5qjt" (UniqueName: "kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt") pod "network-check-target-5zfsf" (UID: "e9766cbc-f4f7-402d-9f9b-15e132d49b71") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:20:21.091549 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:21.091517    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:21.092081 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:21.091648    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zfsf" podUID="e9766cbc-f4f7-402d-9f9b-15e132d49b71"
Aug 05 17:20:21.092081 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:21.091702    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:21.092081 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:21.091819    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-t2vbt" podUID="46b91743-4ee2-4ac1-9779-5b85586520ec"
Aug 05 17:20:22.090909 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.090879    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:22.091065 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:22.091016    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8fkfq" podUID="bfd52bcf-e224-4a4c-8fa2-13c94f985e60"
Aug 05 17:20:22.447633 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.447612    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-37.ec2.internal" event="NodeReady"
Aug 05 17:20:22.448251 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.447765    2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 05 17:20:22.515916 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.515890    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6dqwh"]
Aug 05 17:20:22.517633 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.517616    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-svkcg"]
Aug 05 17:20:22.517743 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.517727    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:22.519332 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.519316    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-svkcg"
Aug 05 17:20:22.520207 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.520186    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 05 17:20:22.520431 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.520415    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 05 17:20:22.520498 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.520485    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t6sjj\""
Aug 05 17:20:22.521819 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.521803    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 05 17:20:22.522661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.522633    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 05 17:20:22.522815 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.522684    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rskhh\""
Aug 05 17:20:22.522815 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.522712    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 05 17:20:22.528037 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.528017    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-svkcg"]
Aug 05 17:20:22.528643 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.528624    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6dqwh"]
Aug 05 17:20:22.567750 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.567726    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"]
Aug 05 17:20:22.569497 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.569484    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"
Aug 05 17:20:22.572006 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.571991    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Aug 05 17:20:22.572161 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.572137    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Aug 05 17:20:22.572221 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.572170    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-42sdw\""
Aug 05 17:20:22.585029 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.585005    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"]
Aug 05 17:20:22.685110 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.685053    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjzxt\" (UniqueName: \"kubernetes.io/projected/07ff7d67-32e8-4c41-846d-ba8fa6400a52-kube-api-access-tjzxt\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:22.685110 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.685093    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhcdf\" (UniqueName: \"kubernetes.io/projected/a5c8f0bd-ce04-46b6-8a24-5fefe12683d3-kube-api-access-xhcdf\") pod \"ingress-canary-svkcg\" (UID: \"a5c8f0bd-ce04-46b6-8a24-5fefe12683d3\") " pod="openshift-ingress-canary/ingress-canary-svkcg"
Aug 05 17:20:22.685281 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.685119    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-7rkk2\" (UID: \"1dd38456-9704-41f6-bb50-f1dab34238ca\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"
Aug 05 17:20:22.685281 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.685137    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/07ff7d67-32e8-4c41-846d-ba8fa6400a52-config-volume\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:22.685281 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.685169    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07ff7d67-32e8-4c41-846d-ba8fa6400a52-tmp-dir\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:22.685281 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.685190    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:22.685281 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.685204    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5c8f0bd-ce04-46b6-8a24-5fefe12683d3-cert\") pod \"ingress-canary-svkcg\" (UID: \"a5c8f0bd-ce04-46b6-8a24-5fefe12683d3\") " pod="openshift-ingress-canary/ingress-canary-svkcg"
Aug 05 17:20:22.685281 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.685219    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1dd38456-9704-41f6-bb50-f1dab34238ca-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-7rkk2\" (UID: \"1dd38456-9704-41f6-bb50-f1dab34238ca\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"
Aug 05 17:20:22.786249 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.786225    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjzxt\" (UniqueName: \"kubernetes.io/projected/07ff7d67-32e8-4c41-846d-ba8fa6400a52-kube-api-access-tjzxt\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:22.786352 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.786264    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhcdf\" (UniqueName: \"kubernetes.io/projected/a5c8f0bd-ce04-46b6-8a24-5fefe12683d3-kube-api-access-xhcdf\") pod \"ingress-canary-svkcg\" (UID: \"a5c8f0bd-ce04-46b6-8a24-5fefe12683d3\") " pod="openshift-ingress-canary/ingress-canary-svkcg"
Aug 05 17:20:22.786352 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.786290    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-7rkk2\" (UID: \"1dd38456-9704-41f6-bb50-f1dab34238ca\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"
Aug 05 17:20:22.786454 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:22.786364    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 05 17:20:22.786454 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.786395    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/07ff7d67-32e8-4c41-846d-ba8fa6400a52-config-volume\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:22.786454 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:22.786411    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert podName:1dd38456-9704-41f6-bb50-f1dab34238ca nodeName:}" failed. No retries permitted until 2026-08-05 17:20:23.286394873 +0000 UTC m=+36.704709609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-7rkk2" (UID: "1dd38456-9704-41f6-bb50-f1dab34238ca") : secret "networking-console-plugin-cert" not found
Aug 05 17:20:22.786454 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.786430    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07ff7d67-32e8-4c41-846d-ba8fa6400a52-tmp-dir\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:22.786640 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.786458    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:22.786640 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:22.786521    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:20:22.786640 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:22.786556    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls podName:07ff7d67-32e8-4c41-846d-ba8fa6400a52 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:23.286544128 +0000 UTC m=+36.704858886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls") pod "dns-default-6dqwh" (UID: "07ff7d67-32e8-4c41-846d-ba8fa6400a52") : secret "dns-default-metrics-tls" not found
Aug 05 17:20:22.786640 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.786583    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5c8f0bd-ce04-46b6-8a24-5fefe12683d3-cert\") pod \"ingress-canary-svkcg\" (UID: \"a5c8f0bd-ce04-46b6-8a24-5fefe12683d3\") " pod="openshift-ingress-canary/ingress-canary-svkcg"
Aug 05 17:20:22.786640 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.786611    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1dd38456-9704-41f6-bb50-f1dab34238ca-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-7rkk2\" (UID: \"1dd38456-9704-41f6-bb50-f1dab34238ca\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"
Aug 05 17:20:22.786818 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.786706    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07ff7d67-32e8-4c41-846d-ba8fa6400a52-tmp-dir\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:22.786883 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.786866    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/07ff7d67-32e8-4c41-846d-ba8fa6400a52-config-volume\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:22.787308 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.787292    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1dd38456-9704-41f6-bb50-f1dab34238ca-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-7rkk2\" (UID: \"1dd38456-9704-41f6-bb50-f1dab34238ca\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"
Aug 05 17:20:22.790783 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.790763    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5c8f0bd-ce04-46b6-8a24-5fefe12683d3-cert\") pod \"ingress-canary-svkcg\" (UID: \"a5c8f0bd-ce04-46b6-8a24-5fefe12683d3\") " pod="openshift-ingress-canary/ingress-canary-svkcg"
Aug 05 17:20:22.799281 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.799261    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhcdf\" (UniqueName: \"kubernetes.io/projected/a5c8f0bd-ce04-46b6-8a24-5fefe12683d3-kube-api-access-xhcdf\") pod \"ingress-canary-svkcg\" (UID: \"a5c8f0bd-ce04-46b6-8a24-5fefe12683d3\") " pod="openshift-ingress-canary/ingress-canary-svkcg"
Aug 05 17:20:22.800132 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.800109    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjzxt\" (UniqueName: \"kubernetes.io/projected/07ff7d67-32e8-4c41-846d-ba8fa6400a52-kube-api-access-tjzxt\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:22.832956 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.832933    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-svkcg"
Aug 05 17:20:22.959635 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:22.959555    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-svkcg"]
Aug 05 17:20:22.962562 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:20:22.962534    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5c8f0bd_ce04_46b6_8a24_5fefe12683d3.slice/crio-5213b6dc9f8d7a0d118582928b18a2c5449cad7404742dbe659c707f951cb98e WatchSource:0}: Error finding container 5213b6dc9f8d7a0d118582928b18a2c5449cad7404742dbe659c707f951cb98e: Status 404 returned error can't find the container with id 5213b6dc9f8d7a0d118582928b18a2c5449cad7404742dbe659c707f951cb98e
Aug 05 17:20:23.090655 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:23.090627    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:23.090655 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:23.090662    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:23.094070 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:23.094048    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 05 17:20:23.094209 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:23.094086    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 05 17:20:23.094209 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:23.094167    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 05 17:20:23.094327 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:23.094258    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bmhgk\""
Aug 05 17:20:23.272791 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:23.272763    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-svkcg" event={"ID":"a5c8f0bd-ce04-46b6-8a24-5fefe12683d3","Type":"ContainerStarted","Data":"5213b6dc9f8d7a0d118582928b18a2c5449cad7404742dbe659c707f951cb98e"}
Aug 05 17:20:23.275324 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:23.275302    2556 generic.go:356] "Generic (PLEG): container finished" podID="4d033317-735c-4582-b54f-f63c0e0663fb" containerID="ee643c59d0d74b3d501990ddbf9ceaf88e56b86e26bf28b7f3a8f42b8c66bedf" exitCode=0
Aug 05 17:20:23.275432 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:23.275337    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzlln" event={"ID":"4d033317-735c-4582-b54f-f63c0e0663fb","Type":"ContainerDied","Data":"ee643c59d0d74b3d501990ddbf9ceaf88e56b86e26bf28b7f3a8f42b8c66bedf"}
Aug 05 17:20:23.291005 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:23.290983    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-7rkk2\" (UID: \"1dd38456-9704-41f6-bb50-f1dab34238ca\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"
Aug 05 17:20:23.291305 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:23.291284    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:23.291532 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:23.291229    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 05 17:20:23.291670 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:23.291660    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert podName:1dd38456-9704-41f6-bb50-f1dab34238ca nodeName:}" failed. No retries permitted until 2026-08-05 17:20:24.291640193 +0000 UTC m=+37.709954933 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-7rkk2" (UID: "1dd38456-9704-41f6-bb50-f1dab34238ca") : secret "networking-console-plugin-cert" not found
Aug 05 17:20:23.291784 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:23.291506    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:20:23.291893 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:23.291817    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls podName:07ff7d67-32e8-4c41-846d-ba8fa6400a52 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:24.291801099 +0000 UTC m=+37.710115848 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls") pod "dns-default-6dqwh" (UID: "07ff7d67-32e8-4c41-846d-ba8fa6400a52") : secret "dns-default-metrics-tls" not found
Aug 05 17:20:24.091197 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:24.091170    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:24.094480 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:24.094273    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 05 17:20:24.094480 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:24.094302    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4rdxm\""
Aug 05 17:20:24.281930 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:24.281903    2556 generic.go:356] "Generic (PLEG): container finished" podID="4d033317-735c-4582-b54f-f63c0e0663fb" containerID="d3ecc8072e0204e7caa7ee7cd6126d5dc56dfa7a002b4aaf4fd5f2ba9822b12b" exitCode=0
Aug 05 17:20:24.282055 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:24.281964    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzlln" event={"ID":"4d033317-735c-4582-b54f-f63c0e0663fb","Type":"ContainerDied","Data":"d3ecc8072e0204e7caa7ee7cd6126d5dc56dfa7a002b4aaf4fd5f2ba9822b12b"}
Aug 05 17:20:24.299212 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:24.299125    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-7rkk2\" (UID: \"1dd38456-9704-41f6-bb50-f1dab34238ca\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"
Aug 05 17:20:24.299361 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:24.299203    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:24.299453 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:24.299275    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 05 17:20:24.299518 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:24.299502    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert podName:1dd38456-9704-41f6-bb50-f1dab34238ca nodeName:}" failed. No retries permitted until 2026-08-05 17:20:26.299484144 +0000 UTC m=+39.717798880 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-7rkk2" (UID: "1dd38456-9704-41f6-bb50-f1dab34238ca") : secret "networking-console-plugin-cert" not found
Aug 05 17:20:24.299815 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:24.299326    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:20:24.299899 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:24.299859    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls podName:07ff7d67-32e8-4c41-846d-ba8fa6400a52 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:26.29983949 +0000 UTC m=+39.718154235 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls") pod "dns-default-6dqwh" (UID: "07ff7d67-32e8-4c41-846d-ba8fa6400a52") : secret "dns-default-metrics-tls" not found
Aug 05 17:20:25.287628 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:25.287447    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzlln" event={"ID":"4d033317-735c-4582-b54f-f63c0e0663fb","Type":"ContainerStarted","Data":"e837af5725ec38880b5580c2e18e5d803b36a3efb8ef140352afca6be1dac58c"}
Aug 05 17:20:25.312880 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:25.312834    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rzlln" podStartSLOduration=9.488086579 podStartE2EDuration="38.312819421s" totalImagesPullingTime="28.824732842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:47 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:25.310932563 +0000 UTC m=+38.729247336" watchObservedRunningTime="2026-08-05 17:20:25.312819421 +0000 UTC m=+38.731134175"
Aug 05 17:20:26.291420 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.291378    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-svkcg" event={"ID":"a5c8f0bd-ce04-46b6-8a24-5fefe12683d3","Type":"ContainerStarted","Data":"5be710a50f38ca169f92abb6589a3879d9e508692fd4adad92c89a8bc05ab8e1"}
Aug 05 17:20:26.308812 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.308768    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-svkcg" podStartSLOduration=1.704238063 podStartE2EDuration="4.308753799s" totalImagesPullingTime="2.604515736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:26.308660646 +0000 UTC m=+39.726975405" watchObservedRunningTime="2026-08-05 17:20:26.308753799 +0000 UTC m=+39.727068534"
Aug 05 17:20:26.315995 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.315962    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-7rkk2\" (UID: \"1dd38456-9704-41f6-bb50-f1dab34238ca\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"
Aug 05 17:20:26.316126 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.316018    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:26.316221 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:26.316128    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:20:26.316221 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:26.316196    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls podName:07ff7d67-32e8-4c41-846d-ba8fa6400a52 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:30.316180511 +0000 UTC m=+43.734495261 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls") pod "dns-default-6dqwh" (UID: "07ff7d67-32e8-4c41-846d-ba8fa6400a52") : secret "dns-default-metrics-tls" not found
Aug 05 17:20:26.316312 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:26.316217    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 05 17:20:26.316312 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:26.316275    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert podName:1dd38456-9704-41f6-bb50-f1dab34238ca nodeName:}" failed. No retries permitted until 2026-08-05 17:20:30.316260509 +0000 UTC m=+43.734575245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-7rkk2" (UID: "1dd38456-9704-41f6-bb50-f1dab34238ca") : secret "networking-console-plugin-cert" not found
Aug 05 17:20:26.716162 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.716132    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-d4wnf"]
Aug 05 17:20:26.737947 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.737927    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:26.743643 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.743617    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 05 17:20:26.744009 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.743989    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 05 17:20:26.745182 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.745054    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 05 17:20:26.745300 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.745283    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qfc7x\""
Aug 05 17:20:26.746344 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.746324    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d4wnf"]
Aug 05 17:20:26.749984 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.749969    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 05 17:20:26.807001 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.806977    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-68kpx"]
Aug 05 17:20:26.827770 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.827752    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-68kpx"
Aug 05 17:20:26.831783 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.831763    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-bcb4d\""
Aug 05 17:20:26.832336 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.832323    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Aug 05 17:20:26.832392 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.832361    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Aug 05 17:20:26.838458 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.838437    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-68kpx"]
Aug 05 17:20:26.920807 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.920780    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:26.920807 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.920808    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc6vp\" (UniqueName: \"kubernetes.io/projected/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-api-access-fc6vp\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:26.920955 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.920843    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0d9598a7-fbf8-4772-859a-6765db90a1b5-crio-socket\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:26.920955 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.920866    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqhns\" (UniqueName: \"kubernetes.io/projected/1395403f-dd78-419d-b2f8-b0fc17073f9d-kube-api-access-qqhns\") pod \"migrator-68bc745d87-68kpx\" (UID: \"1395403f-dd78-419d-b2f8-b0fc17073f9d\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-68kpx"
Aug 05 17:20:26.921026 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.920955    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d9598a7-fbf8-4772-859a-6765db90a1b5-data-volume\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:26.921026 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:26.920984    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:27.021282 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.021213    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:27.021282 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.021246    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc6vp\" (UniqueName: \"kubernetes.io/projected/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-api-access-fc6vp\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:27.021457 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.021293    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0d9598a7-fbf8-4772-859a-6765db90a1b5-crio-socket\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:27.021457 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.021322    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqhns\" (UniqueName: \"kubernetes.io/projected/1395403f-dd78-419d-b2f8-b0fc17073f9d-kube-api-access-qqhns\") pod \"migrator-68bc745d87-68kpx\" (UID: \"1395403f-dd78-419d-b2f8-b0fc17073f9d\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-68kpx"
Aug 05 17:20:27.021457 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:27.021363    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:20:27.021457 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:27.021421    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls podName:0d9598a7-fbf8-4772-859a-6765db90a1b5 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:27.521404476 +0000 UTC m=+40.939719230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d4wnf" (UID: "0d9598a7-fbf8-4772-859a-6765db90a1b5") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:20:27.021645 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.021480    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d9598a7-fbf8-4772-859a-6765db90a1b5-data-volume\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:27.021645 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.021483    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0d9598a7-fbf8-4772-859a-6765db90a1b5-crio-socket\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:27.021645 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.021531    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:27.021645 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:27.021589    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:20:27.021799 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:27.021657    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm podName:0d9598a7-fbf8-4772-859a-6765db90a1b5 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:27.521639575 +0000 UTC m=+40.939954334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d4wnf" (UID: "0d9598a7-fbf8-4772-859a-6765db90a1b5") : configmap "kube-rbac-proxy" not found
Aug 05 17:20:27.021799 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.021772    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d9598a7-fbf8-4772-859a-6765db90a1b5-data-volume\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:27.031296 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.031276    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc6vp\" (UniqueName: \"kubernetes.io/projected/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-api-access-fc6vp\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:27.031296 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.031287    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqhns\" (UniqueName: \"kubernetes.io/projected/1395403f-dd78-419d-b2f8-b0fc17073f9d-kube-api-access-qqhns\") pod \"migrator-68bc745d87-68kpx\" (UID: \"1395403f-dd78-419d-b2f8-b0fc17073f9d\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-68kpx"
Aug 05 17:20:27.136187 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.136164    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-68kpx"
Aug 05 17:20:27.288203 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.288126    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-68kpx"]
Aug 05 17:20:27.291217 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:20:27.291192    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1395403f_dd78_419d_b2f8_b0fc17073f9d.slice/crio-6ddfdd73c1be21b24705bbbb091fc1efcd4b87ce4f330675ec8de15ab6c52843 WatchSource:0}: Error finding container 6ddfdd73c1be21b24705bbbb091fc1efcd4b87ce4f330675ec8de15ab6c52843: Status 404 returned error can't find the container with id 6ddfdd73c1be21b24705bbbb091fc1efcd4b87ce4f330675ec8de15ab6c52843
Aug 05 17:20:27.294941 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.294913    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-68kpx" event={"ID":"1395403f-dd78-419d-b2f8-b0fc17073f9d","Type":"ContainerStarted","Data":"6ddfdd73c1be21b24705bbbb091fc1efcd4b87ce4f330675ec8de15ab6c52843"}
Aug 05 17:20:27.526498 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.526465    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:27.526665 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.526542    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:27.526665 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:27.526609    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:20:27.526665 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:27.526660    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:20:27.526814 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:27.526682    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls podName:0d9598a7-fbf8-4772-859a-6765db90a1b5 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:28.526666676 +0000 UTC m=+41.944981410 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d4wnf" (UID: "0d9598a7-fbf8-4772-859a-6765db90a1b5") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:20:27.526814 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:27.526731    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm podName:0d9598a7-fbf8-4772-859a-6765db90a1b5 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:28.526712788 +0000 UTC m=+41.945027544 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d4wnf" (UID: "0d9598a7-fbf8-4772-859a-6765db90a1b5") : configmap "kube-rbac-proxy" not found
Aug 05 17:20:27.780885 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.780853    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-mppwc"]
Aug 05 17:20:27.806557 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.806534    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-mppwc"]
Aug 05 17:20:27.806725 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.806630    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-mppwc"
Aug 05 17:20:27.809202 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.809173    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Aug 05 17:20:27.809325 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.809266    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Aug 05 17:20:27.810134 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.810113    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Aug 05 17:20:27.810350 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.810337    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Aug 05 17:20:27.810403 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.810366    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-8fg6s\""
Aug 05 17:20:27.930400 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.930375    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgrb2\" (UniqueName: \"kubernetes.io/projected/9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486-kube-api-access-pgrb2\") pod \"service-ca-98c88fbb5-mppwc\" (UID: \"9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486\") " pod="openshift-service-ca/service-ca-98c88fbb5-mppwc"
Aug 05 17:20:27.930507 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.930416    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486-signing-key\") pod \"service-ca-98c88fbb5-mppwc\" (UID: \"9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486\") " pod="openshift-service-ca/service-ca-98c88fbb5-mppwc"
Aug 05 17:20:27.930507 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:27.930450    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486-signing-cabundle\") pod \"service-ca-98c88fbb5-mppwc\" (UID: \"9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486\") " pod="openshift-service-ca/service-ca-98c88fbb5-mppwc"
Aug 05 17:20:28.031565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:28.031502    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgrb2\" (UniqueName: \"kubernetes.io/projected/9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486-kube-api-access-pgrb2\") pod \"service-ca-98c88fbb5-mppwc\" (UID: \"9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486\") " pod="openshift-service-ca/service-ca-98c88fbb5-mppwc"
Aug 05 17:20:28.031565 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:28.031559    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486-signing-key\") pod \"service-ca-98c88fbb5-mppwc\" (UID: \"9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486\") " pod="openshift-service-ca/service-ca-98c88fbb5-mppwc"
Aug 05 17:20:28.031687 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:28.031610    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486-signing-cabundle\") pod \"service-ca-98c88fbb5-mppwc\" (UID: \"9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486\") " pod="openshift-service-ca/service-ca-98c88fbb5-mppwc"
Aug 05 17:20:28.032297 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:28.032280    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486-signing-cabundle\") pod \"service-ca-98c88fbb5-mppwc\" (UID: \"9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486\") " pod="openshift-service-ca/service-ca-98c88fbb5-mppwc"
Aug 05 17:20:28.033827 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:28.033807    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486-signing-key\") pod \"service-ca-98c88fbb5-mppwc\" (UID: \"9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486\") " pod="openshift-service-ca/service-ca-98c88fbb5-mppwc"
Aug 05 17:20:28.040326 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:28.040305    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgrb2\" (UniqueName: \"kubernetes.io/projected/9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486-kube-api-access-pgrb2\") pod \"service-ca-98c88fbb5-mppwc\" (UID: \"9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486\") " pod="openshift-service-ca/service-ca-98c88fbb5-mppwc"
Aug 05 17:20:28.115294 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:28.115269    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-mppwc"
Aug 05 17:20:28.281301 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:28.281270    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-mppwc"]
Aug 05 17:20:28.286235 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:20:28.286203    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a2c001c_6cdc_4d6b_b0c7_5ad6f7230486.slice/crio-0adc0d1f54949f72b373679fa09acb438d2869031b16b47b9dfc6035b731d435 WatchSource:0}: Error finding container 0adc0d1f54949f72b373679fa09acb438d2869031b16b47b9dfc6035b731d435: Status 404 returned error can't find the container with id 0adc0d1f54949f72b373679fa09acb438d2869031b16b47b9dfc6035b731d435
Aug 05 17:20:28.298338 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:28.298311    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-mppwc" event={"ID":"9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486","Type":"ContainerStarted","Data":"0adc0d1f54949f72b373679fa09acb438d2869031b16b47b9dfc6035b731d435"}
Aug 05 17:20:28.457911 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:28.457885    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5z6td_e8d949ff-bbbc-4247-87f0-d9f98e7ec389/dns-node-resolver/0.log"
Aug 05 17:20:28.536716 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:28.536650    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:28.536857 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:28.536720    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:28.536857 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:28.536790    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:20:28.536857 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:28.536806    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:20:28.536857 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:28.536842    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm podName:0d9598a7-fbf8-4772-859a-6765db90a1b5 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:30.536828635 +0000 UTC m=+43.955143370 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d4wnf" (UID: "0d9598a7-fbf8-4772-859a-6765db90a1b5") : configmap "kube-rbac-proxy" not found
Aug 05 17:20:28.537046 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:28.536860    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls podName:0d9598a7-fbf8-4772-859a-6765db90a1b5 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:30.536848053 +0000 UTC m=+43.955162788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d4wnf" (UID: "0d9598a7-fbf8-4772-859a-6765db90a1b5") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:20:29.301819 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:29.301787    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-68kpx" event={"ID":"1395403f-dd78-419d-b2f8-b0fc17073f9d","Type":"ContainerStarted","Data":"547540eb5a3c4cb7d97538b2af22904d6ef5d491605e0eb7d22ae6b9d42b6f87"}
Aug 05 17:20:29.654359 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:29.654328    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l2prc_a264b228-b7f6-4977-b858-820fc604b53c/node-ca/0.log"
Aug 05 17:20:30.306968 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:30.306924    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-68kpx" event={"ID":"1395403f-dd78-419d-b2f8-b0fc17073f9d","Type":"ContainerStarted","Data":"5952dac81838c6deb2b7a39acf5533cd7530a9bb917c2c3eeedd09a361049de9"}
Aug 05 17:20:30.323823 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:30.323761    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-68kpx" podStartSLOduration=2.461210818 podStartE2EDuration="4.323745645s" totalImagesPullingTime="1.862534827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:30.323359015 +0000 UTC m=+43.741673797" watchObservedRunningTime="2026-08-05 17:20:30.323745645 +0000 UTC m=+43.742060407"
Aug 05 17:20:30.347851 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:30.347820    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-7rkk2\" (UID: \"1dd38456-9704-41f6-bb50-f1dab34238ca\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"
Aug 05 17:20:30.347985 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:30.347883    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:30.347985 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:30.347965    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:20:30.348099 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:30.348009    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls podName:07ff7d67-32e8-4c41-846d-ba8fa6400a52 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:38.347997551 +0000 UTC m=+51.766312285 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls") pod "dns-default-6dqwh" (UID: "07ff7d67-32e8-4c41-846d-ba8fa6400a52") : secret "dns-default-metrics-tls" not found
Aug 05 17:20:30.348099 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:30.347965    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 05 17:20:30.348099 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:30.348088    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert podName:1dd38456-9704-41f6-bb50-f1dab34238ca nodeName:}" failed. No retries permitted until 2026-08-05 17:20:38.348072254 +0000 UTC m=+51.766386989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-7rkk2" (UID: "1dd38456-9704-41f6-bb50-f1dab34238ca") : secret "networking-console-plugin-cert" not found
Aug 05 17:20:30.548630 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:30.548595    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:30.548804 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:30.548666    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:30.548804 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:30.548742    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:20:30.548804 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:30.548767    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:20:30.548804 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:30.548792    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm podName:0d9598a7-fbf8-4772-859a-6765db90a1b5 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:34.548780066 +0000 UTC m=+47.967094805 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d4wnf" (UID: "0d9598a7-fbf8-4772-859a-6765db90a1b5") : configmap "kube-rbac-proxy" not found
Aug 05 17:20:30.548966 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:30.548848    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls podName:0d9598a7-fbf8-4772-859a-6765db90a1b5 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:34.548829156 +0000 UTC m=+47.967143912 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d4wnf" (UID: "0d9598a7-fbf8-4772-859a-6765db90a1b5") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:20:30.657944 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:30.657884    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-svkcg_a5c8f0bd-ce04-46b6-8a24-5fefe12683d3/serve-healthcheck-canary/0.log"
Aug 05 17:20:31.311187 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:31.311139    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-mppwc" event={"ID":"9a2c001c-6cdc-4d6b-b0c7-5ad6f7230486","Type":"ContainerStarted","Data":"311ee2259cec57356f9009f03610feefdf303b11121a3c1bb76e04de9eb08562"}
Aug 05 17:20:31.330074 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:31.330032    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-mppwc" podStartSLOduration=1.8555330589999999 podStartE2EDuration="4.330019138s" totalImagesPullingTime="2.474486079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:31.329479347 +0000 UTC m=+44.747794106" watchObservedRunningTime="2026-08-05 17:20:31.330019138 +0000 UTC m=+44.748333896"
Aug 05 17:20:34.583303 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:34.583267    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:34.583815 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:34.583353    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:34.583815 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:34.583404    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:20:34.583815 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:34.583455    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:20:34.583815 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:34.583463    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls podName:0d9598a7-fbf8-4772-859a-6765db90a1b5 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:42.583449363 +0000 UTC m=+56.001764101 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d4wnf" (UID: "0d9598a7-fbf8-4772-859a-6765db90a1b5") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:20:34.583815 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:34.583538    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm podName:0d9598a7-fbf8-4772-859a-6765db90a1b5 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:42.583507961 +0000 UTC m=+56.001822700 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d4wnf" (UID: "0d9598a7-fbf8-4772-859a-6765db90a1b5") : configmap "kube-rbac-proxy" not found
Aug 05 17:20:38.410563 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:38.410524    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:38.411017 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:38.410625    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-7rkk2\" (UID: \"1dd38456-9704-41f6-bb50-f1dab34238ca\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"
Aug 05 17:20:38.411017 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:38.410734    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Aug 05 17:20:38.411017 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:38.410796    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert podName:1dd38456-9704-41f6-bb50-f1dab34238ca nodeName:}" failed. No retries permitted until 2026-08-05 17:20:54.410778438 +0000 UTC m=+67.829093187 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-7rkk2" (UID: "1dd38456-9704-41f6-bb50-f1dab34238ca") : secret "networking-console-plugin-cert" not found
Aug 05 17:20:38.412938 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:38.412911    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/07ff7d67-32e8-4c41-846d-ba8fa6400a52-metrics-tls\") pod \"dns-default-6dqwh\" (UID: \"07ff7d67-32e8-4c41-846d-ba8fa6400a52\") " pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:38.427243 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:38.427219    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:38.560047 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:38.560026    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6dqwh"]
Aug 05 17:20:38.562662 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:20:38.562635    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ff7d67_32e8_4c41_846d_ba8fa6400a52.slice/crio-3d425ebc1d466bd6635e8fc2f08348ed187e653ddbab318dbf5a7798f722c35d WatchSource:0}: Error finding container 3d425ebc1d466bd6635e8fc2f08348ed187e653ddbab318dbf5a7798f722c35d: Status 404 returned error can't find the container with id 3d425ebc1d466bd6635e8fc2f08348ed187e653ddbab318dbf5a7798f722c35d
Aug 05 17:20:39.333229 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:39.333189    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6dqwh" event={"ID":"07ff7d67-32e8-4c41-846d-ba8fa6400a52","Type":"ContainerStarted","Data":"3d425ebc1d466bd6635e8fc2f08348ed187e653ddbab318dbf5a7798f722c35d"}
Aug 05 17:20:41.341316 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:41.341241    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6dqwh" event={"ID":"07ff7d67-32e8-4c41-846d-ba8fa6400a52","Type":"ContainerStarted","Data":"bd92dac3bb2be45b67d600af9f3436d0a62306002a330ee2fee5bfaca331d6f6"}
Aug 05 17:20:41.341316 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:41.341293    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6dqwh" event={"ID":"07ff7d67-32e8-4c41-846d-ba8fa6400a52","Type":"ContainerStarted","Data":"a9b109da547a6c2e439f3bce0feba741fcaa3be61f3e1d96b6ef1a964987a01f"}
Aug 05 17:20:41.341698 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:41.341383    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:41.362489 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:41.362439    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6dqwh" podStartSLOduration=17.169018488 podStartE2EDuration="19.36242611s" totalImagesPullingTime="2.193407622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:41.361112671 +0000 UTC m=+54.779427439" watchObservedRunningTime="2026-08-05 17:20:41.36242611 +0000 UTC m=+54.780740866"
Aug 05 17:20:42.645733 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:42.645696    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:42.646119 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:42.645765    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:42.646119 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:42.645881    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:20:42.646119 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:20:42.645939    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls podName:0d9598a7-fbf8-4772-859a-6765db90a1b5 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:58.645922956 +0000 UTC m=+72.064237710 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d4wnf" (UID: "0d9598a7-fbf8-4772-859a-6765db90a1b5") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:20:42.646281 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:42.646204    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0d9598a7-fbf8-4772-859a-6765db90a1b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:46.263873 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:46.263842    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l9ptq"
Aug 05 17:20:49.159188 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.159141    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74dfc864cd-hmw2j"]
Aug 05 17:20:49.161857 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.161833    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74dfc864cd-hmw2j"
Aug 05 17:20:49.163718 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.163692    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"]
Aug 05 17:20:49.165559 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.165533    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.165685 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.165576    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Aug 05 17:20:49.165685 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.165645    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-2strn\""
Aug 05 17:20:49.166750 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.166730    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Aug 05 17:20:49.166870 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.166733    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Aug 05 17:20:49.166870 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.166772    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Aug 05 17:20:49.167829 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.167813    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Aug 05 17:20:49.168028 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.168011    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Aug 05 17:20:49.168082 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.168066    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Aug 05 17:20:49.168841 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.168583    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Aug 05 17:20:49.175325 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.175307    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74dfc864cd-hmw2j"]
Aug 05 17:20:49.181578 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.181559    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"]
Aug 05 17:20:49.192135 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.192113    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shd6h\" (UniqueName: \"kubernetes.io/projected/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-kube-api-access-shd6h\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.192237 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.192165    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3fb0a674-0ff8-495b-9042-901c31f6d00f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74dfc864cd-hmw2j\" (UID: \"3fb0a674-0ff8-495b-9042-901c31f6d00f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74dfc864cd-hmw2j"
Aug 05 17:20:49.192237 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.192184    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.192237 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.192203    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mxwt\" (UniqueName: \"kubernetes.io/projected/3fb0a674-0ff8-495b-9042-901c31f6d00f-kube-api-access-9mxwt\") pod \"managed-serviceaccount-addon-agent-74dfc864cd-hmw2j\" (UID: \"3fb0a674-0ff8-495b-9042-901c31f6d00f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74dfc864cd-hmw2j"
Aug 05 17:20:49.192237 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.192230    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-ca\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.192365 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.192280    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-hub\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.192365 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.192353    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.192425 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.192372    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.292914 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.292888    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shd6h\" (UniqueName: \"kubernetes.io/projected/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-kube-api-access-shd6h\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.293025 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.292921    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3fb0a674-0ff8-495b-9042-901c31f6d00f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74dfc864cd-hmw2j\" (UID: \"3fb0a674-0ff8-495b-9042-901c31f6d00f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74dfc864cd-hmw2j"
Aug 05 17:20:49.293025 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.292938    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.293025 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.292956    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mxwt\" (UniqueName: \"kubernetes.io/projected/3fb0a674-0ff8-495b-9042-901c31f6d00f-kube-api-access-9mxwt\") pod \"managed-serviceaccount-addon-agent-74dfc864cd-hmw2j\" (UID: \"3fb0a674-0ff8-495b-9042-901c31f6d00f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74dfc864cd-hmw2j"
Aug 05 17:20:49.293025 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.292989    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-ca\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.293256 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.293037    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-hub\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.293256 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.293111    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.293256 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.293135    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.293892 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.293864    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.295737 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.295711    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-hub\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.295829 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.295809    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.295912 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.295891    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3fb0a674-0ff8-495b-9042-901c31f6d00f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74dfc864cd-hmw2j\" (UID: \"3fb0a674-0ff8-495b-9042-901c31f6d00f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74dfc864cd-hmw2j"
Aug 05 17:20:49.295951 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.295899    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-ca\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.296051 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.296032    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.326743 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.326723    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shd6h\" (UniqueName: \"kubernetes.io/projected/0e1c3dcf-05aa-49b7-bf7d-023a4859b67b-kube-api-access-shd6h\") pod \"cluster-proxy-proxy-agent-6ff84847f4-jg6g2\" (UID: \"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.327184 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.327165    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mxwt\" (UniqueName: \"kubernetes.io/projected/3fb0a674-0ff8-495b-9042-901c31f6d00f-kube-api-access-9mxwt\") pod \"managed-serviceaccount-addon-agent-74dfc864cd-hmw2j\" (UID: \"3fb0a674-0ff8-495b-9042-901c31f6d00f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74dfc864cd-hmw2j"
Aug 05 17:20:49.480967 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.480945    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74dfc864cd-hmw2j"
Aug 05 17:20:49.486163 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.486126    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"
Aug 05 17:20:49.612204 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.612172    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74dfc864cd-hmw2j"]
Aug 05 17:20:49.614686 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:20:49.614662    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fb0a674_0ff8_495b_9042_901c31f6d00f.slice/crio-6b33a8a996d93b6b10ae5d0546040af62c7477e80dfc3747630262f26db21862 WatchSource:0}: Error finding container 6b33a8a996d93b6b10ae5d0546040af62c7477e80dfc3747630262f26db21862: Status 404 returned error can't find the container with id 6b33a8a996d93b6b10ae5d0546040af62c7477e80dfc3747630262f26db21862
Aug 05 17:20:49.627676 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:49.627603    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2"]
Aug 05 17:20:49.631076 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:20:49.631053    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e1c3dcf_05aa_49b7_bf7d_023a4859b67b.slice/crio-c4f67ca0bf438fe36f95b8b74f4625b69c01a6e61508d8e557f558e510222c93 WatchSource:0}: Error finding container c4f67ca0bf438fe36f95b8b74f4625b69c01a6e61508d8e557f558e510222c93: Status 404 returned error can't find the container with id c4f67ca0bf438fe36f95b8b74f4625b69c01a6e61508d8e557f558e510222c93
Aug 05 17:20:50.370231 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:50.370190    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2" event={"ID":"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b","Type":"ContainerStarted","Data":"c4f67ca0bf438fe36f95b8b74f4625b69c01a6e61508d8e557f558e510222c93"}
Aug 05 17:20:50.371899 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:50.371861    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74dfc864cd-hmw2j" event={"ID":"3fb0a674-0ff8-495b-9042-901c31f6d00f","Type":"ContainerStarted","Data":"6b33a8a996d93b6b10ae5d0546040af62c7477e80dfc3747630262f26db21862"}
Aug 05 17:20:51.348360 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:51.348327    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6dqwh"
Aug 05 17:20:52.722967 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:52.722931    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:52.725691 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:52.725672    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 05 17:20:52.735973 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:52.735952    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b91743-4ee2-4ac1-9779-5b85586520ec-original-pull-secret\") pod \"global-pull-secret-syncer-t2vbt\" (UID: \"46b91743-4ee2-4ac1-9779-5b85586520ec\") " pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:52.801715 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:52.801677    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-t2vbt"
Aug 05 17:20:52.824203 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:52.824177    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs\") pod \"network-metrics-daemon-8fkfq\" (UID: \"bfd52bcf-e224-4a4c-8fa2-13c94f985e60\") " pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:52.827673 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:52.827653    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 05 17:20:52.837306 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:52.837286    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd52bcf-e224-4a4c-8fa2-13c94f985e60-metrics-certs\") pod \"network-metrics-daemon-8fkfq\" (UID: \"bfd52bcf-e224-4a4c-8fa2-13c94f985e60\") " pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:52.904172 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:52.904136    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4rdxm\""
Aug 05 17:20:52.912282 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:52.912260    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8fkfq"
Aug 05 17:20:52.925425 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:52.925398    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5qjt\" (UniqueName: \"kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt\") pod \"network-check-target-5zfsf\" (UID: \"e9766cbc-f4f7-402d-9f9b-15e132d49b71\") " pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:52.928365 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:52.928340    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 05 17:20:52.938855 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:52.938831    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 05 17:20:52.949883 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:52.949861    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5qjt\" (UniqueName: \"kubernetes.io/projected/e9766cbc-f4f7-402d-9f9b-15e132d49b71-kube-api-access-j5qjt\") pod \"network-check-target-5zfsf\" (UID: \"e9766cbc-f4f7-402d-9f9b-15e132d49b71\") " pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:53.108258 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:53.108183    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bmhgk\""
Aug 05 17:20:53.116164 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:53.116128    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:20:54.439006 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:54.438970    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-7rkk2\" (UID: \"1dd38456-9704-41f6-bb50-f1dab34238ca\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"
Aug 05 17:20:54.441057 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:54.441037    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1dd38456-9704-41f6-bb50-f1dab34238ca-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-7rkk2\" (UID: \"1dd38456-9704-41f6-bb50-f1dab34238ca\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"
Aug 05 17:20:54.679744 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:54.679584    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-42sdw\""
Aug 05 17:20:54.690062 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:54.688844    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"
Aug 05 17:20:54.748679 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:54.748649    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5zfsf"]
Aug 05 17:20:54.752192 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:20:54.752130    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9766cbc_f4f7_402d_9f9b_15e132d49b71.slice/crio-47f8c3b6b833f236cdc957cace5508e0469cc2b20b9d78601fed02cc97a3d59d WatchSource:0}: Error finding container 47f8c3b6b833f236cdc957cace5508e0469cc2b20b9d78601fed02cc97a3d59d: Status 404 returned error can't find the container with id 47f8c3b6b833f236cdc957cace5508e0469cc2b20b9d78601fed02cc97a3d59d
Aug 05 17:20:54.765946 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:54.765803    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8fkfq"]
Aug 05 17:20:54.853805 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:54.853780    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2"]
Aug 05 17:20:54.856511 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:20:54.856484    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dd38456_9704_41f6_bb50_f1dab34238ca.slice/crio-a7d4eceffa6f6976d7e8ce900fbe31c0f6a0aa197f1d06e5ab5b38c4bc0db027 WatchSource:0}: Error finding container a7d4eceffa6f6976d7e8ce900fbe31c0f6a0aa197f1d06e5ab5b38c4bc0db027: Status 404 returned error can't find the container with id a7d4eceffa6f6976d7e8ce900fbe31c0f6a0aa197f1d06e5ab5b38c4bc0db027
Aug 05 17:20:54.968854 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:54.968828    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-t2vbt"]
Aug 05 17:20:54.972987 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:20:54.972963    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46b91743_4ee2_4ac1_9779_5b85586520ec.slice/crio-14aee2ab8bf43b2bacacfbd22ea35e93c42dbe1bcf5c95e20163821616f17180 WatchSource:0}: Error finding container 14aee2ab8bf43b2bacacfbd22ea35e93c42dbe1bcf5c95e20163821616f17180: Status 404 returned error can't find the container with id 14aee2ab8bf43b2bacacfbd22ea35e93c42dbe1bcf5c95e20163821616f17180
Aug 05 17:20:55.390473 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:55.390392    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74dfc864cd-hmw2j" event={"ID":"3fb0a674-0ff8-495b-9042-901c31f6d00f","Type":"ContainerStarted","Data":"5ad120ff453c42fac3622eb3825c933298959a043f37a4a42f5b9a8ff3af1549"}
Aug 05 17:20:55.392165 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:55.392093    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2" event={"ID":"1dd38456-9704-41f6-bb50-f1dab34238ca","Type":"ContainerStarted","Data":"a7d4eceffa6f6976d7e8ce900fbe31c0f6a0aa197f1d06e5ab5b38c4bc0db027"}
Aug 05 17:20:55.393488 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:55.393460    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8fkfq" event={"ID":"bfd52bcf-e224-4a4c-8fa2-13c94f985e60","Type":"ContainerStarted","Data":"4c3d6761256c9c6b58052caa1bf959b9419207b606c01dafbf8ba6ca8d41e475"}
Aug 05 17:20:55.396196 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:55.396169    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2" event={"ID":"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b","Type":"ContainerStarted","Data":"e1448b1cef142aa4122f68e5cd68e246a90958060d0fdc6ca4c385595121b531"}
Aug 05 17:20:55.396304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:55.396217    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2" event={"ID":"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b","Type":"ContainerStarted","Data":"156020b5f1e5bec36f88a6733ea9fe73ece60ce33be0692eb9808b749552bdad"}
Aug 05 17:20:55.396304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:55.396236    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2" event={"ID":"0e1c3dcf-05aa-49b7-bf7d-023a4859b67b","Type":"ContainerStarted","Data":"09e7379655b411bc5967021cc20e123b63435216d5ade7526f1dc769d57635d6"}
Aug 05 17:20:55.397836 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:55.397770    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t2vbt" event={"ID":"46b91743-4ee2-4ac1-9779-5b85586520ec","Type":"ContainerStarted","Data":"14aee2ab8bf43b2bacacfbd22ea35e93c42dbe1bcf5c95e20163821616f17180"}
Aug 05 17:20:55.399718 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:55.399695    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5zfsf" event={"ID":"e9766cbc-f4f7-402d-9f9b-15e132d49b71","Type":"ContainerStarted","Data":"47f8c3b6b833f236cdc957cace5508e0469cc2b20b9d78601fed02cc97a3d59d"}
Aug 05 17:20:55.437110 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:55.436756    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74dfc864cd-hmw2j" podStartSLOduration=1.461701756 podStartE2EDuration="6.436737502s" totalImagesPullingTime="4.975035746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:55.409064983 +0000 UTC m=+68.827379741" watchObservedRunningTime="2026-08-05 17:20:55.436737502 +0000 UTC m=+68.855052260"
Aug 05 17:20:57.412644 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:57.412603    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8fkfq" event={"ID":"bfd52bcf-e224-4a4c-8fa2-13c94f985e60","Type":"ContainerStarted","Data":"f06eb55ca8495fcb31b7ecc6e91b834eb8cd64725aa326ef30c30744dd9ac983"}
Aug 05 17:20:57.413079 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:57.412672    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8fkfq" event={"ID":"bfd52bcf-e224-4a4c-8fa2-13c94f985e60","Type":"ContainerStarted","Data":"21cbdaf1453e45c56486ab1c0a9aeefede7e4ad74ee0c75adfbdbf0fe14e6836"}
Aug 05 17:20:57.414433 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:57.414397    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2" event={"ID":"1dd38456-9704-41f6-bb50-f1dab34238ca","Type":"ContainerStarted","Data":"364a4d724d6642c96c0e9cc909b709bc546c5d0d8844e63081d4b6dbd1fcf5d0"}
Aug 05 17:20:57.436010 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:57.435965    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8fkfq" podStartSLOduration=68.477749324 podStartE2EDuration="1m10.435949516s" totalImagesPullingTime="1.958200192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:19:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:57.435694659 +0000 UTC m=+70.854009418" watchObservedRunningTime="2026-08-05 17:20:57.435949516 +0000 UTC m=+70.854264271"
Aug 05 17:20:57.436424 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:57.436386    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6ff84847f4-jg6g2" podStartSLOduration=3.460669969 podStartE2EDuration="8.436374406s" totalImagesPullingTime="4.975704437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:55.437670481 +0000 UTC m=+68.855985240" watchObservedRunningTime="2026-08-05 17:20:57.436374406 +0000 UTC m=+70.854689163"
Aug 05 17:20:57.452726 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:57.452682    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-7rkk2" podStartSLOduration=33.574686174 podStartE2EDuration="35.452673381s" totalImagesPullingTime="1.877987207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:57.451356167 +0000 UTC m=+70.869670975" watchObservedRunningTime="2026-08-05 17:20:57.452673381 +0000 UTC m=+70.870988160"
Aug 05 17:20:58.672722 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:58.672681    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:58.675471 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:58.675437    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0d9598a7-fbf8-4772-859a-6765db90a1b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d4wnf\" (UID: \"0d9598a7-fbf8-4772-859a-6765db90a1b5\") " pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:58.849699 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:58.849663    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qfc7x\""
Aug 05 17:20:58.858130 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:58.858100    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d4wnf"
Aug 05 17:20:59.641905 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:20:59.641873    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d4wnf"]
Aug 05 17:20:59.645961 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:20:59.645935    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d9598a7_fbf8_4772_859a_6765db90a1b5.slice/crio-e333d43dd9fed30b799b55b74690697458e1033921024e3bb02424467036b7b6 WatchSource:0}: Error finding container e333d43dd9fed30b799b55b74690697458e1033921024e3bb02424467036b7b6: Status 404 returned error can't find the container with id e333d43dd9fed30b799b55b74690697458e1033921024e3bb02424467036b7b6
Aug 05 17:21:00.427197 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:00.427139    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d4wnf" event={"ID":"0d9598a7-fbf8-4772-859a-6765db90a1b5","Type":"ContainerStarted","Data":"fbe48b7df532370f6a06f453e5c4a7f16c890f25fba65f6c730e91a24dba6aa3"}
Aug 05 17:21:00.427612 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:00.427227    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d4wnf" event={"ID":"0d9598a7-fbf8-4772-859a-6765db90a1b5","Type":"ContainerStarted","Data":"6e93682a8525e0ac5f0a8e4b67533b89332792514d3f5c7799fc1940722dde0a"}
Aug 05 17:21:00.427612 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:00.427253    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d4wnf" event={"ID":"0d9598a7-fbf8-4772-859a-6765db90a1b5","Type":"ContainerStarted","Data":"e333d43dd9fed30b799b55b74690697458e1033921024e3bb02424467036b7b6"}
Aug 05 17:21:00.428582 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:00.428556    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-t2vbt" event={"ID":"46b91743-4ee2-4ac1-9779-5b85586520ec","Type":"ContainerStarted","Data":"15d3e0d3df09894757f00c9d5eb597873532c6ead6b3c13b7319ab9e292d1d8d"}
Aug 05 17:21:00.429972 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:00.429903    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5zfsf" event={"ID":"e9766cbc-f4f7-402d-9f9b-15e132d49b71","Type":"ContainerStarted","Data":"f0992108237d46fa4d083945c4fe01c6f87f3d51c3bb874d12fc6f51d723fc24"}
Aug 05 17:21:00.430103 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:00.430088    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:21:00.444328 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:00.444279    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-t2vbt" podStartSLOduration=68.89919418 podStartE2EDuration="1m13.444266059s" totalImagesPullingTime="4.545071879s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:00.443497885 +0000 UTC m=+73.861812642" watchObservedRunningTime="2026-08-05 17:21:00.444266059 +0000 UTC m=+73.862580816"
Aug 05 17:21:00.460571 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:00.460538    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5zfsf" podStartSLOduration=68.703311752 podStartE2EDuration="1m13.460525828s" totalImagesPullingTime="4.757214076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:19:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:00.460042618 +0000 UTC m=+73.878357398" watchObservedRunningTime="2026-08-05 17:21:00.460525828 +0000 UTC m=+73.878840587"
Aug 05 17:21:02.439764 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:02.439725    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d4wnf" event={"ID":"0d9598a7-fbf8-4772-859a-6765db90a1b5","Type":"ContainerStarted","Data":"64390e87148a7810867fdd10e9deefa478dc73506039d178f742c238fa6f0123"}
Aug 05 17:21:04.194989 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.194919    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-d4wnf" podStartSLOduration=36.224450301 podStartE2EDuration="38.194894761s" totalImagesPullingTime="1.97044446s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:20:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:02.461856909 +0000 UTC m=+75.880171683" watchObservedRunningTime="2026-08-05 17:21:04.194894761 +0000 UTC m=+77.613209520"
Aug 05 17:21:04.195511 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.195484    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5fslc"]
Aug 05 17:21:04.198671 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.198654    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.201846 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.201603    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 05 17:21:04.201846 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.201693    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kbf9b\""
Aug 05 17:21:04.201846 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.201735    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 05 17:21:04.201846 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.201692    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 05 17:21:04.202692 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.202664    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 05 17:21:04.202824 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.202667    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 05 17:21:04.202824 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.202724    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 05 17:21:04.309486 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.309455    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-tls\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.309486 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.309496    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.309718 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.309526    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-wtmp\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.309718 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.309610    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac88833f-ac4d-4f2c-a09c-e4259988f433-sys\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.309718 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.309651    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-accelerators-collector-config\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.309718 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.309682    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-textfile\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.309718 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.309708    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ac88833f-ac4d-4f2c-a09c-e4259988f433-root\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.309961 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.309741    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdnm6\" (UniqueName: \"kubernetes.io/projected/ac88833f-ac4d-4f2c-a09c-e4259988f433-kube-api-access-tdnm6\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.309961 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.309771    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac88833f-ac4d-4f2c-a09c-e4259988f433-metrics-client-ca\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.411057 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.411024    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac88833f-ac4d-4f2c-a09c-e4259988f433-sys\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.411229 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.411104    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-accelerators-collector-config\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.411229 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.411138    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-textfile\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.411229 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.411211    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac88833f-ac4d-4f2c-a09c-e4259988f433-sys\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.411380 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.411228    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ac88833f-ac4d-4f2c-a09c-e4259988f433-root\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.411380 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.411265    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdnm6\" (UniqueName: \"kubernetes.io/projected/ac88833f-ac4d-4f2c-a09c-e4259988f433-kube-api-access-tdnm6\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.411380 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.411293    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac88833f-ac4d-4f2c-a09c-e4259988f433-metrics-client-ca\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.411380 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.411358    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-tls\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.411586 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.411382    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.411586 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.411409    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-wtmp\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.411686 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.411581    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-wtmp\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.411790 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:21:04.411756    2556 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Aug 05 17:21:04.411839 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:21:04.411819    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-tls podName:ac88833f-ac4d-4f2c-a09c-e4259988f433 nodeName:}" failed. No retries permitted until 2026-08-05 17:21:04.911800941 +0000 UTC m=+78.330115689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-tls") pod "node-exporter-5fslc" (UID: "ac88833f-ac4d-4f2c-a09c-e4259988f433") : secret "node-exporter-tls" not found
Aug 05 17:21:04.411839 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.411815    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-textfile\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.411951 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.411814    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-accelerators-collector-config\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.412038 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.412017    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ac88833f-ac4d-4f2c-a09c-e4259988f433-root\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.412251 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.412228    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac88833f-ac4d-4f2c-a09c-e4259988f433-metrics-client-ca\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.414389 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.414358    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.424697 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.424667    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdnm6\" (UniqueName: \"kubernetes.io/projected/ac88833f-ac4d-4f2c-a09c-e4259988f433-kube-api-access-tdnm6\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.915193 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.915138    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-tls\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:04.917351 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:04.917321    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ac88833f-ac4d-4f2c-a09c-e4259988f433-node-exporter-tls\") pod \"node-exporter-5fslc\" (UID: \"ac88833f-ac4d-4f2c-a09c-e4259988f433\") " pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:05.110178 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:05.110136    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5fslc"
Aug 05 17:21:05.120825 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:21:05.120795    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac88833f_ac4d_4f2c_a09c_e4259988f433.slice/crio-39d6c4f1a2c7a2cdabb7f75ea6fe594e110bb8a463a0acc1a09a5e20b110d858 WatchSource:0}: Error finding container 39d6c4f1a2c7a2cdabb7f75ea6fe594e110bb8a463a0acc1a09a5e20b110d858: Status 404 returned error can't find the container with id 39d6c4f1a2c7a2cdabb7f75ea6fe594e110bb8a463a0acc1a09a5e20b110d858
Aug 05 17:21:05.451552 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:05.451519    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5fslc" event={"ID":"ac88833f-ac4d-4f2c-a09c-e4259988f433","Type":"ContainerStarted","Data":"39d6c4f1a2c7a2cdabb7f75ea6fe594e110bb8a463a0acc1a09a5e20b110d858"}
Aug 05 17:21:06.458718 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:06.458635    2556 generic.go:356] "Generic (PLEG): container finished" podID="ac88833f-ac4d-4f2c-a09c-e4259988f433" containerID="87124219d1a2d7239150d349f43685b2cd420079311440e2ad8ce5baeafd263c" exitCode=0
Aug 05 17:21:06.458718 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:06.458684    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5fslc" event={"ID":"ac88833f-ac4d-4f2c-a09c-e4259988f433","Type":"ContainerDied","Data":"87124219d1a2d7239150d349f43685b2cd420079311440e2ad8ce5baeafd263c"}
Aug 05 17:21:07.464884 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:07.464850    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5fslc" event={"ID":"ac88833f-ac4d-4f2c-a09c-e4259988f433","Type":"ContainerStarted","Data":"896b546dd61b9f553024aec2562cad30ade05489f94a886633584fd12e45f8fa"}
Aug 05 17:21:07.465316 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:07.464906    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5fslc" event={"ID":"ac88833f-ac4d-4f2c-a09c-e4259988f433","Type":"ContainerStarted","Data":"15d3af6840fca165e6449fb7415b63c044006bea2c4c84b346c6135665ee8d7a"}
Aug 05 17:21:07.493937 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:07.493885    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5fslc" podStartSLOduration=2.4460589710000002 podStartE2EDuration="3.493871508s" totalImagesPullingTime="986.157758ms" totalInitContainerRuntime="61.654779ms" isStatefulPod=true podCreationTimestamp="2026-08-05 17:21:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:07.492664925 +0000 UTC m=+80.910979683" watchObservedRunningTime="2026-08-05 17:21:07.493871508 +0000 UTC m=+80.912186264"
Aug 05 17:21:31.437663 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:31.437629    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5zfsf"
Aug 05 17:21:42.815253 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:21:42.815222    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-svkcg_a5c8f0bd-ce04-46b6-8a24-5fefe12683d3/serve-healthcheck-canary/0.log"
Aug 05 17:24:46.932121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:46.932096    2556 kubelet.go:1648] "Image garbage collection succeeded"
Aug 05 17:24:54.673735 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:54.673696    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-gpbft"]
Aug 05 17:24:54.675513 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:54.675495    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-gpbft"
Aug 05 17:24:54.677820 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:54.677793    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Aug 05 17:24:54.677919 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:54.677806    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Aug 05 17:24:54.678732 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:54.678718    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-8b2m4\""
Aug 05 17:24:54.686281 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:54.686261    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-gpbft"]
Aug 05 17:24:54.703392 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:54.703368    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6731e589-6e1b-41e9-affa-e90e9e2fa907-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-gpbft\" (UID: \"6731e589-6e1b-41e9-affa-e90e9e2fa907\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gpbft"
Aug 05 17:24:54.703492 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:54.703416    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d4g9\" (UniqueName: \"kubernetes.io/projected/6731e589-6e1b-41e9-affa-e90e9e2fa907-kube-api-access-9d4g9\") pod \"cert-manager-webhook-6486ffd997-gpbft\" (UID: \"6731e589-6e1b-41e9-affa-e90e9e2fa907\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gpbft"
Aug 05 17:24:54.803685 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:54.803663    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6731e589-6e1b-41e9-affa-e90e9e2fa907-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-gpbft\" (UID: \"6731e589-6e1b-41e9-affa-e90e9e2fa907\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gpbft"
Aug 05 17:24:54.803767 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:54.803703    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d4g9\" (UniqueName: \"kubernetes.io/projected/6731e589-6e1b-41e9-affa-e90e9e2fa907-kube-api-access-9d4g9\") pod \"cert-manager-webhook-6486ffd997-gpbft\" (UID: \"6731e589-6e1b-41e9-affa-e90e9e2fa907\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gpbft"
Aug 05 17:24:54.811437 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:54.811412    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6731e589-6e1b-41e9-affa-e90e9e2fa907-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-gpbft\" (UID: \"6731e589-6e1b-41e9-affa-e90e9e2fa907\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gpbft"
Aug 05 17:24:54.811532 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:54.811511    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d4g9\" (UniqueName: \"kubernetes.io/projected/6731e589-6e1b-41e9-affa-e90e9e2fa907-kube-api-access-9d4g9\") pod \"cert-manager-webhook-6486ffd997-gpbft\" (UID: \"6731e589-6e1b-41e9-affa-e90e9e2fa907\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gpbft"
Aug 05 17:24:54.984600 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:54.984574    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-gpbft"
Aug 05 17:24:55.105918 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:55.105890    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-gpbft"]
Aug 05 17:24:55.109927 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:24:55.109903    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6731e589_6e1b_41e9_affa_e90e9e2fa907.slice/crio-a36c75765526239b9f4587cf7b0f991e3b8e87bc4983dc459485ce4233ae8f50 WatchSource:0}: Error finding container a36c75765526239b9f4587cf7b0f991e3b8e87bc4983dc459485ce4233ae8f50: Status 404 returned error can't find the container with id a36c75765526239b9f4587cf7b0f991e3b8e87bc4983dc459485ce4233ae8f50
Aug 05 17:24:55.111815 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:55.111799    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:24:55.184503 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:55.184474    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-gpbft" event={"ID":"6731e589-6e1b-41e9-affa-e90e9e2fa907","Type":"ContainerStarted","Data":"a36c75765526239b9f4587cf7b0f991e3b8e87bc4983dc459485ce4233ae8f50"}
Aug 05 17:24:59.199950 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:59.199910    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-gpbft" event={"ID":"6731e589-6e1b-41e9-affa-e90e9e2fa907","Type":"ContainerStarted","Data":"6e3ed0a5baf31af19a7c9a663f14687dcb6f76dc2532c6c9e16368948034b91a"}
Aug 05 17:24:59.200344 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:59.200035    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-gpbft"
Aug 05 17:24:59.216007 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:24:59.215960    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-gpbft" podStartSLOduration=2.168686986 podStartE2EDuration="5.215948702s" totalImagesPullingTime="3.047261716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:24:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:24:59.214959153 +0000 UTC m=+312.633273912" watchObservedRunningTime="2026-08-05 17:24:59.215948702 +0000 UTC m=+312.634263458"
Aug 05 17:25:05.205097 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:05.205067    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-gpbft"
Aug 05 17:25:08.236385 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:08.236355    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-r7w5g"]
Aug 05 17:25:08.238393 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:08.238377    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-r7w5g"
Aug 05 17:25:08.240699 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:08.240677    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-qdd29\""
Aug 05 17:25:08.249164 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:08.249127    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-r7w5g"]
Aug 05 17:25:08.293763 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:08.293743    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fbwg\" (UniqueName: \"kubernetes.io/projected/c438839b-913a-44f6-b0a2-a69c8673e574-kube-api-access-7fbwg\") pod \"cert-manager-55b4d954d5-r7w5g\" (UID: \"c438839b-913a-44f6-b0a2-a69c8673e574\") " pod="cert-manager/cert-manager-55b4d954d5-r7w5g"
Aug 05 17:25:08.293877 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:08.293791    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c438839b-913a-44f6-b0a2-a69c8673e574-bound-sa-token\") pod \"cert-manager-55b4d954d5-r7w5g\" (UID: \"c438839b-913a-44f6-b0a2-a69c8673e574\") " pod="cert-manager/cert-manager-55b4d954d5-r7w5g"
Aug 05 17:25:08.394891 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:08.394865    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fbwg\" (UniqueName: \"kubernetes.io/projected/c438839b-913a-44f6-b0a2-a69c8673e574-kube-api-access-7fbwg\") pod \"cert-manager-55b4d954d5-r7w5g\" (UID: \"c438839b-913a-44f6-b0a2-a69c8673e574\") " pod="cert-manager/cert-manager-55b4d954d5-r7w5g"
Aug 05 17:25:08.394976 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:08.394913    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c438839b-913a-44f6-b0a2-a69c8673e574-bound-sa-token\") pod \"cert-manager-55b4d954d5-r7w5g\" (UID: \"c438839b-913a-44f6-b0a2-a69c8673e574\") " pod="cert-manager/cert-manager-55b4d954d5-r7w5g"
Aug 05 17:25:08.403770 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:08.403749    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c438839b-913a-44f6-b0a2-a69c8673e574-bound-sa-token\") pod \"cert-manager-55b4d954d5-r7w5g\" (UID: \"c438839b-913a-44f6-b0a2-a69c8673e574\") " pod="cert-manager/cert-manager-55b4d954d5-r7w5g"
Aug 05 17:25:08.403974 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:08.403956    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fbwg\" (UniqueName: \"kubernetes.io/projected/c438839b-913a-44f6-b0a2-a69c8673e574-kube-api-access-7fbwg\") pod \"cert-manager-55b4d954d5-r7w5g\" (UID: \"c438839b-913a-44f6-b0a2-a69c8673e574\") " pod="cert-manager/cert-manager-55b4d954d5-r7w5g"
Aug 05 17:25:08.547464 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:08.547407    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-r7w5g"
Aug 05 17:25:08.669689 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:08.669662    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-r7w5g"]
Aug 05 17:25:08.672669 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:25:08.672640    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc438839b_913a_44f6_b0a2_a69c8673e574.slice/crio-2ac8181813c9a785efd58b57bafb0d237595b5329de1da836ddba51e443cbe0d WatchSource:0}: Error finding container 2ac8181813c9a785efd58b57bafb0d237595b5329de1da836ddba51e443cbe0d: Status 404 returned error can't find the container with id 2ac8181813c9a785efd58b57bafb0d237595b5329de1da836ddba51e443cbe0d
Aug 05 17:25:09.232575 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:09.232543    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-r7w5g" event={"ID":"c438839b-913a-44f6-b0a2-a69c8673e574","Type":"ContainerStarted","Data":"320c8df2d578b1e61c7aa592e7729e8f52bf154f8ce0df2d7a1b1f9354eae2e2"}
Aug 05 17:25:09.232575 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:09.232582    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-r7w5g" event={"ID":"c438839b-913a-44f6-b0a2-a69c8673e574","Type":"ContainerStarted","Data":"2ac8181813c9a785efd58b57bafb0d237595b5329de1da836ddba51e443cbe0d"}
Aug 05 17:25:09.249485 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:09.249442    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-r7w5g" podStartSLOduration=1.249427364 podStartE2EDuration="1.249427364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:25:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:25:09.247576286 +0000 UTC m=+322.665891134" watchObservedRunningTime="2026-08-05 17:25:09.249427364 +0000 UTC m=+322.667742120"
Aug 05 17:25:23.163753 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.163720    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"]
Aug 05 17:25:23.165761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.165746    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.168730 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.168702    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Aug 05 17:25:23.168852 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.168840    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Aug 05 17:25:23.168918 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.168878    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Aug 05 17:25:23.174648 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.169709    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9vhvx\""
Aug 05 17:25:23.174648 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.169959    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Aug 05 17:25:23.180624 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.180597    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"]
Aug 05 17:25:23.291822 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.291794    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbdd8ac3-41d3-4941-a2ec-a764a2640b37-tmp\") pod \"opendatahub-operator-controller-manager-58c579b484-dzh4p\" (UID: \"cbdd8ac3-41d3-4941-a2ec-a764a2640b37\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.291968 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.291827    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cbdd8ac3-41d3-4941-a2ec-a764a2640b37-manifests\") pod \"opendatahub-operator-controller-manager-58c579b484-dzh4p\" (UID: \"cbdd8ac3-41d3-4941-a2ec-a764a2640b37\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.291968 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.291852    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cbdd8ac3-41d3-4941-a2ec-a764a2640b37-webhook-cert\") pod \"opendatahub-operator-controller-manager-58c579b484-dzh4p\" (UID: \"cbdd8ac3-41d3-4941-a2ec-a764a2640b37\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.291968 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.291877    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cbdd8ac3-41d3-4941-a2ec-a764a2640b37-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58c579b484-dzh4p\" (UID: \"cbdd8ac3-41d3-4941-a2ec-a764a2640b37\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.291968 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.291911    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rf82\" (UniqueName: \"kubernetes.io/projected/cbdd8ac3-41d3-4941-a2ec-a764a2640b37-kube-api-access-5rf82\") pod \"opendatahub-operator-controller-manager-58c579b484-dzh4p\" (UID: \"cbdd8ac3-41d3-4941-a2ec-a764a2640b37\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.392188 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.392162    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cbdd8ac3-41d3-4941-a2ec-a764a2640b37-webhook-cert\") pod \"opendatahub-operator-controller-manager-58c579b484-dzh4p\" (UID: \"cbdd8ac3-41d3-4941-a2ec-a764a2640b37\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.392350 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.392196    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cbdd8ac3-41d3-4941-a2ec-a764a2640b37-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58c579b484-dzh4p\" (UID: \"cbdd8ac3-41d3-4941-a2ec-a764a2640b37\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.392350 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.392225    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rf82\" (UniqueName: \"kubernetes.io/projected/cbdd8ac3-41d3-4941-a2ec-a764a2640b37-kube-api-access-5rf82\") pod \"opendatahub-operator-controller-manager-58c579b484-dzh4p\" (UID: \"cbdd8ac3-41d3-4941-a2ec-a764a2640b37\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.392350 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.392261    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbdd8ac3-41d3-4941-a2ec-a764a2640b37-tmp\") pod \"opendatahub-operator-controller-manager-58c579b484-dzh4p\" (UID: \"cbdd8ac3-41d3-4941-a2ec-a764a2640b37\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.392350 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.392298    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cbdd8ac3-41d3-4941-a2ec-a764a2640b37-manifests\") pod \"opendatahub-operator-controller-manager-58c579b484-dzh4p\" (UID: \"cbdd8ac3-41d3-4941-a2ec-a764a2640b37\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.392729 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.392707    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cbdd8ac3-41d3-4941-a2ec-a764a2640b37-manifests\") pod \"opendatahub-operator-controller-manager-58c579b484-dzh4p\" (UID: \"cbdd8ac3-41d3-4941-a2ec-a764a2640b37\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.394575 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.394550    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbdd8ac3-41d3-4941-a2ec-a764a2640b37-tmp\") pod \"opendatahub-operator-controller-manager-58c579b484-dzh4p\" (UID: \"cbdd8ac3-41d3-4941-a2ec-a764a2640b37\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.394688 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.394668    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cbdd8ac3-41d3-4941-a2ec-a764a2640b37-webhook-cert\") pod \"opendatahub-operator-controller-manager-58c579b484-dzh4p\" (UID: \"cbdd8ac3-41d3-4941-a2ec-a764a2640b37\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.394866 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.394847    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cbdd8ac3-41d3-4941-a2ec-a764a2640b37-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58c579b484-dzh4p\" (UID: \"cbdd8ac3-41d3-4941-a2ec-a764a2640b37\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.399952 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.399933    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rf82\" (UniqueName: \"kubernetes.io/projected/cbdd8ac3-41d3-4941-a2ec-a764a2640b37-kube-api-access-5rf82\") pod \"opendatahub-operator-controller-manager-58c579b484-dzh4p\" (UID: \"cbdd8ac3-41d3-4941-a2ec-a764a2640b37\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.477957 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.477934    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:23.599636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:23.599613    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"]
Aug 05 17:25:23.601764 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:25:23.601740    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbdd8ac3_41d3_4941_a2ec_a764a2640b37.slice/crio-e5d5ec288d9dd7ef532bed8769e67a42b78ff818e942a33fb674f43217e3a584 WatchSource:0}: Error finding container e5d5ec288d9dd7ef532bed8769e67a42b78ff818e942a33fb674f43217e3a584: Status 404 returned error can't find the container with id e5d5ec288d9dd7ef532bed8769e67a42b78ff818e942a33fb674f43217e3a584
Aug 05 17:25:24.282213 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:24.282184    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p" event={"ID":"cbdd8ac3-41d3-4941-a2ec-a764a2640b37","Type":"ContainerStarted","Data":"e5d5ec288d9dd7ef532bed8769e67a42b78ff818e942a33fb674f43217e3a584"}
Aug 05 17:25:27.296566 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:27.296525    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p" event={"ID":"cbdd8ac3-41d3-4941-a2ec-a764a2640b37","Type":"ContainerStarted","Data":"51bd195e3b083130c8c1ba7bdb312a5c206f088d7ceb37cc192eb82636869001"}
Aug 05 17:25:28.301811 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:28.301775    2556 generic.go:356] "Generic (PLEG): container finished" podID="cbdd8ac3-41d3-4941-a2ec-a764a2640b37" containerID="51bd195e3b083130c8c1ba7bdb312a5c206f088d7ceb37cc192eb82636869001" exitCode=0
Aug 05 17:25:28.301811 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:28.301814    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p" event={"ID":"cbdd8ac3-41d3-4941-a2ec-a764a2640b37","Type":"ContainerDied","Data":"51bd195e3b083130c8c1ba7bdb312a5c206f088d7ceb37cc192eb82636869001"}
Aug 05 17:25:29.306518 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:29.306479    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p" event={"ID":"cbdd8ac3-41d3-4941-a2ec-a764a2640b37","Type":"ContainerStarted","Data":"0d2975514f002da06c19dedb3cb2bf70fbfb3addce59d53da480295e10ae32d7"}
Aug 05 17:25:29.306969 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:29.306618    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:29.328848 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:29.328800    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p" podStartSLOduration=1.659756494 podStartE2EDuration="6.328785095s" totalImagesPullingTime="3.59504627s" totalInitContainerRuntime="1.073982331s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:25:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:25:29.327071639 +0000 UTC m=+342.745386414" watchObservedRunningTime="2026-08-05 17:25:29.328785095 +0000 UTC m=+342.747099851"
Aug 05 17:25:40.313879 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:40.313806    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-dzh4p"
Aug 05 17:25:43.269922 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.269887    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm"]
Aug 05 17:25:43.272463 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.272446    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm"
Aug 05 17:25:43.275130 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.275112    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Aug 05 17:25:43.276312 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.276252    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 05 17:25:43.276312 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.276267    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Aug 05 17:25:43.276312 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.276255    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 05 17:25:43.276532 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.276378    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8ttfc\""
Aug 05 17:25:43.301353 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.301335    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm"]
Aug 05 17:25:43.337104 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.337073    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2686161c-63d4-4b55-9f0f-0583e8ce7f56-tmp\") pod \"kube-auth-proxy-7c99fc649-hq2hm\" (UID: \"2686161c-63d4-4b55-9f0f-0583e8ce7f56\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm"
Aug 05 17:25:43.337215 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.337135    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7ng4\" (UniqueName: \"kubernetes.io/projected/2686161c-63d4-4b55-9f0f-0583e8ce7f56-kube-api-access-m7ng4\") pod \"kube-auth-proxy-7c99fc649-hq2hm\" (UID: \"2686161c-63d4-4b55-9f0f-0583e8ce7f56\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm"
Aug 05 17:25:43.337259 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.337224    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2686161c-63d4-4b55-9f0f-0583e8ce7f56-tls-certs\") pod \"kube-auth-proxy-7c99fc649-hq2hm\" (UID: \"2686161c-63d4-4b55-9f0f-0583e8ce7f56\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm"
Aug 05 17:25:43.438445 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.438410    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7ng4\" (UniqueName: \"kubernetes.io/projected/2686161c-63d4-4b55-9f0f-0583e8ce7f56-kube-api-access-m7ng4\") pod \"kube-auth-proxy-7c99fc649-hq2hm\" (UID: \"2686161c-63d4-4b55-9f0f-0583e8ce7f56\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm"
Aug 05 17:25:43.438635 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.438465    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2686161c-63d4-4b55-9f0f-0583e8ce7f56-tls-certs\") pod \"kube-auth-proxy-7c99fc649-hq2hm\" (UID: \"2686161c-63d4-4b55-9f0f-0583e8ce7f56\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm"
Aug 05 17:25:43.438635 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.438493    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2686161c-63d4-4b55-9f0f-0583e8ce7f56-tmp\") pod \"kube-auth-proxy-7c99fc649-hq2hm\" (UID: \"2686161c-63d4-4b55-9f0f-0583e8ce7f56\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm"
Aug 05 17:25:43.440691 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.440661    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2686161c-63d4-4b55-9f0f-0583e8ce7f56-tmp\") pod \"kube-auth-proxy-7c99fc649-hq2hm\" (UID: \"2686161c-63d4-4b55-9f0f-0583e8ce7f56\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm"
Aug 05 17:25:43.440941 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.440919    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2686161c-63d4-4b55-9f0f-0583e8ce7f56-tls-certs\") pod \"kube-auth-proxy-7c99fc649-hq2hm\" (UID: \"2686161c-63d4-4b55-9f0f-0583e8ce7f56\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm"
Aug 05 17:25:43.457614 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.457588    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7ng4\" (UniqueName: \"kubernetes.io/projected/2686161c-63d4-4b55-9f0f-0583e8ce7f56-kube-api-access-m7ng4\") pod \"kube-auth-proxy-7c99fc649-hq2hm\" (UID: \"2686161c-63d4-4b55-9f0f-0583e8ce7f56\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm"
Aug 05 17:25:43.582658 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.582569    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm"
Aug 05 17:25:43.771093 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:43.771065    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm"]
Aug 05 17:25:43.771995 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:25:43.771967    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2686161c_63d4_4b55_9f0f_0583e8ce7f56.slice/crio-70f6e4758ad3345f3977173a08822f93f916baf414f2b64996343c4a207f75f5 WatchSource:0}: Error finding container 70f6e4758ad3345f3977173a08822f93f916baf414f2b64996343c4a207f75f5: Status 404 returned error can't find the container with id 70f6e4758ad3345f3977173a08822f93f916baf414f2b64996343c4a207f75f5
Aug 05 17:25:44.357702 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:44.357662    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm" event={"ID":"2686161c-63d4-4b55-9f0f-0583e8ce7f56","Type":"ContainerStarted","Data":"70f6e4758ad3345f3977173a08822f93f916baf414f2b64996343c4a207f75f5"}
Aug 05 17:25:44.808452 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:44.807218    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-h69xq"]
Aug 05 17:25:44.809534 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:44.809511    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-h69xq"
Aug 05 17:25:44.813070 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:44.813039    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Aug 05 17:25:44.825798 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:44.825690    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-h69xq"]
Aug 05 17:25:44.951647 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:44.951608    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jx59\" (UniqueName: \"kubernetes.io/projected/40aca00f-bca1-4a21-a4c3-b18adf59745e-kube-api-access-6jx59\") pod \"model-serving-api-54d89bd5fc-h69xq\" (UID: \"40aca00f-bca1-4a21-a4c3-b18adf59745e\") " pod="opendatahub/model-serving-api-54d89bd5fc-h69xq"
Aug 05 17:25:44.951826 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:44.951694    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40aca00f-bca1-4a21-a4c3-b18adf59745e-tls-certs\") pod \"model-serving-api-54d89bd5fc-h69xq\" (UID: \"40aca00f-bca1-4a21-a4c3-b18adf59745e\") " pod="opendatahub/model-serving-api-54d89bd5fc-h69xq"
Aug 05 17:25:45.053142 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.053099    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jx59\" (UniqueName: \"kubernetes.io/projected/40aca00f-bca1-4a21-a4c3-b18adf59745e-kube-api-access-6jx59\") pod \"model-serving-api-54d89bd5fc-h69xq\" (UID: \"40aca00f-bca1-4a21-a4c3-b18adf59745e\") " pod="opendatahub/model-serving-api-54d89bd5fc-h69xq"
Aug 05 17:25:45.053350 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.053215    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40aca00f-bca1-4a21-a4c3-b18adf59745e-tls-certs\") pod \"model-serving-api-54d89bd5fc-h69xq\" (UID: \"40aca00f-bca1-4a21-a4c3-b18adf59745e\") " pod="opendatahub/model-serving-api-54d89bd5fc-h69xq"
Aug 05 17:25:45.053350 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:25:45.053344    2556 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Aug 05 17:25:45.053467 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:25:45.053403    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40aca00f-bca1-4a21-a4c3-b18adf59745e-tls-certs podName:40aca00f-bca1-4a21-a4c3-b18adf59745e nodeName:}" failed. No retries permitted until 2026-08-05 17:25:45.553383112 +0000 UTC m=+358.971697847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/40aca00f-bca1-4a21-a4c3-b18adf59745e-tls-certs") pod "model-serving-api-54d89bd5fc-h69xq" (UID: "40aca00f-bca1-4a21-a4c3-b18adf59745e") : secret "model-serving-api-tls" not found
Aug 05 17:25:45.130709 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.130515    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jx59\" (UniqueName: \"kubernetes.io/projected/40aca00f-bca1-4a21-a4c3-b18adf59745e-kube-api-access-6jx59\") pod \"model-serving-api-54d89bd5fc-h69xq\" (UID: \"40aca00f-bca1-4a21-a4c3-b18adf59745e\") " pod="opendatahub/model-serving-api-54d89bd5fc-h69xq"
Aug 05 17:25:45.406037 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.405958    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"]
Aug 05 17:25:45.409727 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.409692    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.414062 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.414033    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Aug 05 17:25:45.414214 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.414033    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-j989w\""
Aug 05 17:25:45.414278 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.414198    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Aug 05 17:25:45.416087 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.415428    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Aug 05 17:25:45.416087 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.415620    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Aug 05 17:25:45.432796 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.432768    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"]
Aug 05 17:25:45.557908 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.557873    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.558091 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.557919    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.558091 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.557958    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgsdz\" (UniqueName: \"kubernetes.io/projected/78635888-e00e-4ba4-9e78-b3491f583716-kube-api-access-pgsdz\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.558091 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.558006    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40aca00f-bca1-4a21-a4c3-b18adf59745e-tls-certs\") pod \"model-serving-api-54d89bd5fc-h69xq\" (UID: \"40aca00f-bca1-4a21-a4c3-b18adf59745e\") " pod="opendatahub/model-serving-api-54d89bd5fc-h69xq"
Aug 05 17:25:45.558091 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.558042    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78635888-e00e-4ba4-9e78-b3491f583716-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.558091 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.558067    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78635888-e00e-4ba4-9e78-b3491f583716-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.558391 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.558100    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78635888-e00e-4ba4-9e78-b3491f583716-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.558391 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.558124    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.561283 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.561234    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40aca00f-bca1-4a21-a4c3-b18adf59745e-tls-certs\") pod \"model-serving-api-54d89bd5fc-h69xq\" (UID: \"40aca00f-bca1-4a21-a4c3-b18adf59745e\") " pod="opendatahub/model-serving-api-54d89bd5fc-h69xq"
Aug 05 17:25:45.658750 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.658664    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.658750 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.658706    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.658976 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.658822    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgsdz\" (UniqueName: \"kubernetes.io/projected/78635888-e00e-4ba4-9e78-b3491f583716-kube-api-access-pgsdz\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.658976 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.658890    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78635888-e00e-4ba4-9e78-b3491f583716-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.658976 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.658917    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78635888-e00e-4ba4-9e78-b3491f583716-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.658976 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.658958    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78635888-e00e-4ba4-9e78-b3491f583716-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.659224 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.658983    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.659701 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.659663    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78635888-e00e-4ba4-9e78-b3491f583716-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.661569 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.661543    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.661985 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.661959    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78635888-e00e-4ba4-9e78-b3491f583716-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.662182 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.662118    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.662324 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.662302    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.668796 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.668771    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78635888-e00e-4ba4-9e78-b3491f583716-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.669074 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.669055    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgsdz\" (UniqueName: \"kubernetes.io/projected/78635888-e00e-4ba4-9e78-b3491f583716-kube-api-access-pgsdz\") pod \"istiod-openshift-gateway-7b7995d5cd-bcxxr\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.723811 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.723784    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:45.728568 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:45.728530    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-h69xq"
Aug 05 17:25:46.893380 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:46.893354    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-h69xq"]
Aug 05 17:25:46.894906 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:25:46.894879    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40aca00f_bca1_4a21_a4c3_b18adf59745e.slice/crio-c6ade9d230fa0015e8bf3175fa9e017468299b0115404c01c82bd717b1c38c99 WatchSource:0}: Error finding container c6ade9d230fa0015e8bf3175fa9e017468299b0115404c01c82bd717b1c38c99: Status 404 returned error can't find the container with id c6ade9d230fa0015e8bf3175fa9e017468299b0115404c01c82bd717b1c38c99
Aug 05 17:25:46.924539 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:46.924514    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"]
Aug 05 17:25:46.927926 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:25:46.927899    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78635888_e00e_4ba4_9e78_b3491f583716.slice/crio-d93f8c3a73d63a940f106381a989cf02fe0f4235deb6e1304ef61628e1c7a139 WatchSource:0}: Error finding container d93f8c3a73d63a940f106381a989cf02fe0f4235deb6e1304ef61628e1c7a139: Status 404 returned error can't find the container with id d93f8c3a73d63a940f106381a989cf02fe0f4235deb6e1304ef61628e1c7a139
Aug 05 17:25:47.370952 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:47.370916    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-h69xq" event={"ID":"40aca00f-bca1-4a21-a4c3-b18adf59745e","Type":"ContainerStarted","Data":"c6ade9d230fa0015e8bf3175fa9e017468299b0115404c01c82bd717b1c38c99"}
Aug 05 17:25:47.372586 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:47.372555    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm" event={"ID":"2686161c-63d4-4b55-9f0f-0583e8ce7f56","Type":"ContainerStarted","Data":"77ce2f7b2d79efea62004de9e4e7c8101fcca268108e3ae8129f5edc5df5283d"}
Aug 05 17:25:47.373804 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:47.373773    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr" event={"ID":"78635888-e00e-4ba4-9e78-b3491f583716","Type":"ContainerStarted","Data":"d93f8c3a73d63a940f106381a989cf02fe0f4235deb6e1304ef61628e1c7a139"}
Aug 05 17:25:47.407515 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:47.407465    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c99fc649-hq2hm" podStartSLOduration=1.411341866 podStartE2EDuration="4.407447774s" totalImagesPullingTime="2.996105908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:25:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:25:47.405936244 +0000 UTC m=+360.824251000" watchObservedRunningTime="2026-08-05 17:25:47.407447774 +0000 UTC m=+360.825762533"
Aug 05 17:25:49.384840 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:49.384791    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-h69xq" event={"ID":"40aca00f-bca1-4a21-a4c3-b18adf59745e","Type":"ContainerStarted","Data":"81e6f94dd0c3f3ec7b97c15fc4e6a7a362c5fc36089d5c778897d82952216ccd"}
Aug 05 17:25:49.385326 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:49.384918    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-h69xq"
Aug 05 17:25:49.404220 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:49.404173    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-h69xq" podStartSLOduration=3.495173346 podStartE2EDuration="5.404141819s" totalImagesPullingTime="1.908968473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:25:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:25:49.402567466 +0000 UTC m=+362.820882247" watchObservedRunningTime="2026-08-05 17:25:49.404141819 +0000 UTC m=+362.822456577"
Aug 05 17:25:49.828743 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:49.828706    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Aug 05 17:25:49.828852 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:49.828775    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Aug 05 17:25:50.394666 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:50.394631    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr" event={"ID":"78635888-e00e-4ba4-9e78-b3491f583716","Type":"ContainerStarted","Data":"3bc284eabb853c6b13e3dbd0e53964e81b717479ceb0d1d97fa8b82cddc9603c"}
Aug 05 17:25:50.395113 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:50.394815    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:50.396550 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:50.396524    2556 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-bcxxr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 05 17:25:50.396661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:50.396578    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr" podUID="78635888-e00e-4ba4-9e78-b3491f583716" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 05 17:25:50.416378 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:50.416335    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr" podStartSLOduration=2.518196404 podStartE2EDuration="5.416320471s" totalImagesPullingTime="2.898124067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:25:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:25:50.414646296 +0000 UTC m=+363.832961074" watchObservedRunningTime="2026-08-05 17:25:50.416320471 +0000 UTC m=+363.834635231"
Aug 05 17:25:51.399315 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:51.399286    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:54.820378 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:54.820341    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-48244"]
Aug 05 17:25:54.822838 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:54.822807    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj"]
Aug 05 17:25:54.822939 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:54.822917    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-48244"
Aug 05 17:25:54.825505 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:54.825480    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj"
Aug 05 17:25:54.825638 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:54.825516    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-5ccl7\""
Aug 05 17:25:54.825844 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:54.825827    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Aug 05 17:25:54.827845 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:54.827809    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Aug 05 17:25:54.828934 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:54.828916    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-g2nmb\""
Aug 05 17:25:54.840218 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:54.840197    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-48244"]
Aug 05 17:25:54.843060 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:54.843033    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj"]
Aug 05 17:25:54.935312 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:54.935275    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pwpl\" (UniqueName: \"kubernetes.io/projected/93bced0f-67fa-4d9e-b335-1970769ca3e5-kube-api-access-9pwpl\") pod \"llmisvc-controller-manager-777c6bd7d5-hd9qj\" (UID: \"93bced0f-67fa-4d9e-b335-1970769ca3e5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj"
Aug 05 17:25:54.935455 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:54.935322    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qghxv\" (UniqueName: \"kubernetes.io/projected/a86647ff-9a52-4ae9-85fa-421e108a9805-kube-api-access-qghxv\") pod \"kserve-controller-manager-d6f4b65b8-48244\" (UID: \"a86647ff-9a52-4ae9-85fa-421e108a9805\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-48244"
Aug 05 17:25:54.935455 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:54.935393    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/93bced0f-67fa-4d9e-b335-1970769ca3e5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hd9qj\" (UID: \"93bced0f-67fa-4d9e-b335-1970769ca3e5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj"
Aug 05 17:25:54.935455 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:54.935426    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a86647ff-9a52-4ae9-85fa-421e108a9805-cert\") pod \"kserve-controller-manager-d6f4b65b8-48244\" (UID: \"a86647ff-9a52-4ae9-85fa-421e108a9805\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-48244"
Aug 05 17:25:55.036303 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:55.036269    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qghxv\" (UniqueName: \"kubernetes.io/projected/a86647ff-9a52-4ae9-85fa-421e108a9805-kube-api-access-qghxv\") pod \"kserve-controller-manager-d6f4b65b8-48244\" (UID: \"a86647ff-9a52-4ae9-85fa-421e108a9805\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-48244"
Aug 05 17:25:55.036468 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:55.036313    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/93bced0f-67fa-4d9e-b335-1970769ca3e5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hd9qj\" (UID: \"93bced0f-67fa-4d9e-b335-1970769ca3e5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj"
Aug 05 17:25:55.036468 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:55.036339    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a86647ff-9a52-4ae9-85fa-421e108a9805-cert\") pod \"kserve-controller-manager-d6f4b65b8-48244\" (UID: \"a86647ff-9a52-4ae9-85fa-421e108a9805\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-48244"
Aug 05 17:25:55.036468 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:55.036382    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pwpl\" (UniqueName: \"kubernetes.io/projected/93bced0f-67fa-4d9e-b335-1970769ca3e5-kube-api-access-9pwpl\") pod \"llmisvc-controller-manager-777c6bd7d5-hd9qj\" (UID: \"93bced0f-67fa-4d9e-b335-1970769ca3e5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj"
Aug 05 17:25:55.036468 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:25:55.036427    2556 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Aug 05 17:25:55.036682 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:25:55.036497    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93bced0f-67fa-4d9e-b335-1970769ca3e5-cert podName:93bced0f-67fa-4d9e-b335-1970769ca3e5 nodeName:}" failed. No retries permitted until 2026-08-05 17:25:55.536478901 +0000 UTC m=+368.954793638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/93bced0f-67fa-4d9e-b335-1970769ca3e5-cert") pod "llmisvc-controller-manager-777c6bd7d5-hd9qj" (UID: "93bced0f-67fa-4d9e-b335-1970769ca3e5") : secret "llmisvc-webhook-server-cert" not found
Aug 05 17:25:55.038738 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:55.038716    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a86647ff-9a52-4ae9-85fa-421e108a9805-cert\") pod \"kserve-controller-manager-d6f4b65b8-48244\" (UID: \"a86647ff-9a52-4ae9-85fa-421e108a9805\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-48244"
Aug 05 17:25:55.047681 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:55.047659    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qghxv\" (UniqueName: \"kubernetes.io/projected/a86647ff-9a52-4ae9-85fa-421e108a9805-kube-api-access-qghxv\") pod \"kserve-controller-manager-d6f4b65b8-48244\" (UID: \"a86647ff-9a52-4ae9-85fa-421e108a9805\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-48244"
Aug 05 17:25:55.047786 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:55.047722    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pwpl\" (UniqueName: \"kubernetes.io/projected/93bced0f-67fa-4d9e-b335-1970769ca3e5-kube-api-access-9pwpl\") pod \"llmisvc-controller-manager-777c6bd7d5-hd9qj\" (UID: \"93bced0f-67fa-4d9e-b335-1970769ca3e5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj"
Aug 05 17:25:55.138443 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:55.138381    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-48244"
Aug 05 17:25:55.284166 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:55.284122    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-48244"]
Aug 05 17:25:55.287347 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:25:55.287317    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda86647ff_9a52_4ae9_85fa_421e108a9805.slice/crio-ec00148cb5eaa157a344c6e4dcc47955410479f05c7611c57093579b051e1d18 WatchSource:0}: Error finding container ec00148cb5eaa157a344c6e4dcc47955410479f05c7611c57093579b051e1d18: Status 404 returned error can't find the container with id ec00148cb5eaa157a344c6e4dcc47955410479f05c7611c57093579b051e1d18
Aug 05 17:25:55.420043 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:55.419968    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-48244" event={"ID":"a86647ff-9a52-4ae9-85fa-421e108a9805","Type":"ContainerStarted","Data":"ec00148cb5eaa157a344c6e4dcc47955410479f05c7611c57093579b051e1d18"}
Aug 05 17:25:55.541806 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:55.541777    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/93bced0f-67fa-4d9e-b335-1970769ca3e5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hd9qj\" (UID: \"93bced0f-67fa-4d9e-b335-1970769ca3e5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj"
Aug 05 17:25:55.544057 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:55.544039    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/93bced0f-67fa-4d9e-b335-1970769ca3e5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hd9qj\" (UID: \"93bced0f-67fa-4d9e-b335-1970769ca3e5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj"
Aug 05 17:25:55.747863 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:55.747837    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj"
Aug 05 17:25:55.898918 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:55.898892    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj"]
Aug 05 17:25:55.900847 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:25:55.900818    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod93bced0f_67fa_4d9e_b335_1970769ca3e5.slice/crio-beb50be2884718baecccbcc4023a525fa9f4c7fa2b86786c9425063ee94bbe0b WatchSource:0}: Error finding container beb50be2884718baecccbcc4023a525fa9f4c7fa2b86786c9425063ee94bbe0b: Status 404 returned error can't find the container with id beb50be2884718baecccbcc4023a525fa9f4c7fa2b86786c9425063ee94bbe0b
Aug 05 17:25:56.426820 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:56.426786    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj" event={"ID":"93bced0f-67fa-4d9e-b335-1970769ca3e5","Type":"ContainerStarted","Data":"beb50be2884718baecccbcc4023a525fa9f4c7fa2b86786c9425063ee94bbe0b"}
Aug 05 17:25:57.768506 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:57.768477    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"]
Aug 05 17:25:57.768954 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:57.768690    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr" podUID="78635888-e00e-4ba4-9e78-b3491f583716" containerName="discovery" containerID="cri-o://3bc284eabb853c6b13e3dbd0e53964e81b717479ceb0d1d97fa8b82cddc9603c" gracePeriod=30
Aug 05 17:25:58.163104 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.163080    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:58.263854 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.263827    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78635888-e00e-4ba4-9e78-b3491f583716-istio-csr-ca-configmap\") pod \"78635888-e00e-4ba4-9e78-b3491f583716\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") "
Aug 05 17:25:58.264003 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.263886    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-istio-csr-dns-cert\") pod \"78635888-e00e-4ba4-9e78-b3491f583716\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") "
Aug 05 17:25:58.264003 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.263913    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgsdz\" (UniqueName: \"kubernetes.io/projected/78635888-e00e-4ba4-9e78-b3491f583716-kube-api-access-pgsdz\") pod \"78635888-e00e-4ba4-9e78-b3491f583716\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") "
Aug 05 17:25:58.264003 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.263931    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78635888-e00e-4ba4-9e78-b3491f583716-local-certs\") pod \"78635888-e00e-4ba4-9e78-b3491f583716\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") "
Aug 05 17:25:58.264003 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.263990    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-cacerts\") pod \"78635888-e00e-4ba4-9e78-b3491f583716\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") "
Aug 05 17:25:58.264222 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.264006    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-istio-kubeconfig\") pod \"78635888-e00e-4ba4-9e78-b3491f583716\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") "
Aug 05 17:25:58.264222 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.264029    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78635888-e00e-4ba4-9e78-b3491f583716-istio-token\") pod \"78635888-e00e-4ba4-9e78-b3491f583716\" (UID: \"78635888-e00e-4ba4-9e78-b3491f583716\") "
Aug 05 17:25:58.264393 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.264256    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78635888-e00e-4ba4-9e78-b3491f583716-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "78635888-e00e-4ba4-9e78-b3491f583716" (UID: "78635888-e00e-4ba4-9e78-b3491f583716"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:25:58.266453 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.266425    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78635888-e00e-4ba4-9e78-b3491f583716-istio-token" (OuterVolumeSpecName: "istio-token") pod "78635888-e00e-4ba4-9e78-b3491f583716" (UID: "78635888-e00e-4ba4-9e78-b3491f583716"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:25:58.266626 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.266597    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-cacerts" (OuterVolumeSpecName: "cacerts") pod "78635888-e00e-4ba4-9e78-b3491f583716" (UID: "78635888-e00e-4ba4-9e78-b3491f583716"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:25:58.266729 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.266628    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78635888-e00e-4ba4-9e78-b3491f583716-local-certs" (OuterVolumeSpecName: "local-certs") pod "78635888-e00e-4ba4-9e78-b3491f583716" (UID: "78635888-e00e-4ba4-9e78-b3491f583716"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:25:58.266808 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.266731    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78635888-e00e-4ba4-9e78-b3491f583716-kube-api-access-pgsdz" (OuterVolumeSpecName: "kube-api-access-pgsdz") pod "78635888-e00e-4ba4-9e78-b3491f583716" (UID: "78635888-e00e-4ba4-9e78-b3491f583716"). InnerVolumeSpecName "kube-api-access-pgsdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:25:58.266808 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.266780    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "78635888-e00e-4ba4-9e78-b3491f583716" (UID: "78635888-e00e-4ba4-9e78-b3491f583716"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:25:58.266985 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.266927    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "78635888-e00e-4ba4-9e78-b3491f583716" (UID: "78635888-e00e-4ba4-9e78-b3491f583716"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:25:58.365065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.365010    2556 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-cacerts\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:25:58.365217 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.365062    2556 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-istio-kubeconfig\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:25:58.365217 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.365086    2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78635888-e00e-4ba4-9e78-b3491f583716-istio-token\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:25:58.365217 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.365113    2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78635888-e00e-4ba4-9e78-b3491f583716-istio-csr-ca-configmap\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:25:58.365217 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.365133    2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78635888-e00e-4ba4-9e78-b3491f583716-istio-csr-dns-cert\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:25:58.365217 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.365145    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgsdz\" (UniqueName: \"kubernetes.io/projected/78635888-e00e-4ba4-9e78-b3491f583716-kube-api-access-pgsdz\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:25:58.365217 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.365171    2556 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78635888-e00e-4ba4-9e78-b3491f583716-local-certs\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:25:58.436580 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.436547    2556 generic.go:356] "Generic (PLEG): container finished" podID="78635888-e00e-4ba4-9e78-b3491f583716" containerID="3bc284eabb853c6b13e3dbd0e53964e81b717479ceb0d1d97fa8b82cddc9603c" exitCode=0
Aug 05 17:25:58.436713 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.436597    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"
Aug 05 17:25:58.436713 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.436621    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr" event={"ID":"78635888-e00e-4ba4-9e78-b3491f583716","Type":"ContainerDied","Data":"3bc284eabb853c6b13e3dbd0e53964e81b717479ceb0d1d97fa8b82cddc9603c"}
Aug 05 17:25:58.436713 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.436665    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr" event={"ID":"78635888-e00e-4ba4-9e78-b3491f583716","Type":"ContainerDied","Data":"d93f8c3a73d63a940f106381a989cf02fe0f4235deb6e1304ef61628e1c7a139"}
Aug 05 17:25:58.436713 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.436689    2556 scope.go:117] "RemoveContainer" containerID="3bc284eabb853c6b13e3dbd0e53964e81b717479ceb0d1d97fa8b82cddc9603c"
Aug 05 17:25:58.438305 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.438274    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-48244" event={"ID":"a86647ff-9a52-4ae9-85fa-421e108a9805","Type":"ContainerStarted","Data":"1c8444b7a3eea917ee1d204ee859c357141498306976875a57fa7af2e8f6b72a"}
Aug 05 17:25:58.438417 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.438384    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-48244"
Aug 05 17:25:58.445051 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.445036    2556 scope.go:117] "RemoveContainer" containerID="3bc284eabb853c6b13e3dbd0e53964e81b717479ceb0d1d97fa8b82cddc9603c"
Aug 05 17:25:58.445317 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:25:58.445294    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bc284eabb853c6b13e3dbd0e53964e81b717479ceb0d1d97fa8b82cddc9603c\": container with ID starting with 3bc284eabb853c6b13e3dbd0e53964e81b717479ceb0d1d97fa8b82cddc9603c not found: ID does not exist" containerID="3bc284eabb853c6b13e3dbd0e53964e81b717479ceb0d1d97fa8b82cddc9603c"
Aug 05 17:25:58.445394 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.445328    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bc284eabb853c6b13e3dbd0e53964e81b717479ceb0d1d97fa8b82cddc9603c"} err="failed to get container status \"3bc284eabb853c6b13e3dbd0e53964e81b717479ceb0d1d97fa8b82cddc9603c\": rpc error: code = NotFound desc = could not find container \"3bc284eabb853c6b13e3dbd0e53964e81b717479ceb0d1d97fa8b82cddc9603c\": container with ID starting with 3bc284eabb853c6b13e3dbd0e53964e81b717479ceb0d1d97fa8b82cddc9603c not found: ID does not exist"
Aug 05 17:25:58.468637 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.468208    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-48244" podStartSLOduration=1.703087636 podStartE2EDuration="4.468189949s" totalImagesPullingTime="2.765102313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:25:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:25:58.467047098 +0000 UTC m=+371.885361854" watchObservedRunningTime="2026-08-05 17:25:58.468189949 +0000 UTC m=+371.886504708"
Aug 05 17:25:58.485625 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.485599    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"]
Aug 05 17:25:58.493061 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:58.493039    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bcxxr"]
Aug 05 17:25:59.095878 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:25:59.095847    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78635888-e00e-4ba4-9e78-b3491f583716" path="/var/lib/kubelet/pods/78635888-e00e-4ba4-9e78-b3491f583716/volumes"
Aug 05 17:26:00.399115 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:26:00.399095    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-h69xq"
Aug 05 17:26:01.452945 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:26:01.452910    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj" event={"ID":"93bced0f-67fa-4d9e-b335-1970769ca3e5","Type":"ContainerStarted","Data":"ba421bedcd55b0d717dc7b4d732820787705303997413cafeaa01919ef220b19"}
Aug 05 17:26:01.453360 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:26:01.453005    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj"
Aug 05 17:26:01.486693 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:26:01.486642    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj" podStartSLOduration=3.013124715 podStartE2EDuration="7.486625648s" totalImagesPullingTime="4.473500933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:25:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:26:01.486038337 +0000 UTC m=+374.904353093" watchObservedRunningTime="2026-08-05 17:26:01.486625648 +0000 UTC m=+374.904940406"
Aug 05 17:26:29.448202 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:26:29.448171    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-48244"
Aug 05 17:26:32.459224 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:26:32.459194    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hd9qj"
Aug 05 17:27:00.344192 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:00.344140    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"]
Aug 05 17:27:00.344606 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:00.344491    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78635888-e00e-4ba4-9e78-b3491f583716" containerName="discovery"
Aug 05 17:27:00.344606 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:00.344505    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="78635888-e00e-4ba4-9e78-b3491f583716" containerName="discovery"
Aug 05 17:27:00.344606 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:00.344554    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="78635888-e00e-4ba4-9e78-b3491f583716" containerName="discovery"
Aug 05 17:27:00.346927 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:00.346911    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"
Aug 05 17:27:00.349582 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:00.349561    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 05 17:27:00.349707 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:00.349561    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-nh7bs\""
Aug 05 17:27:00.350270 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:00.350256    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 05 17:27:00.365145 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:00.365118    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"]
Aug 05 17:27:00.384439 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:00.384418    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g6bc\" (UniqueName: \"kubernetes.io/projected/069a1514-aa31-44e0-8192-2c7e29e089e1-kube-api-access-5g6bc\") pod \"limitador-operator-controller-manager-6c549b6d85-ws5wh\" (UID: \"069a1514-aa31-44e0-8192-2c7e29e089e1\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"
Aug 05 17:27:00.484694 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:00.484660    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g6bc\" (UniqueName: \"kubernetes.io/projected/069a1514-aa31-44e0-8192-2c7e29e089e1-kube-api-access-5g6bc\") pod \"limitador-operator-controller-manager-6c549b6d85-ws5wh\" (UID: \"069a1514-aa31-44e0-8192-2c7e29e089e1\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"
Aug 05 17:27:00.501557 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:00.501521    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g6bc\" (UniqueName: \"kubernetes.io/projected/069a1514-aa31-44e0-8192-2c7e29e089e1-kube-api-access-5g6bc\") pod \"limitador-operator-controller-manager-6c549b6d85-ws5wh\" (UID: \"069a1514-aa31-44e0-8192-2c7e29e089e1\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"
Aug 05 17:27:00.657141 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:00.657065    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"
Aug 05 17:27:00.780473 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:00.780443    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"]
Aug 05 17:27:00.783897 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:27:00.783870    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod069a1514_aa31_44e0_8192_2c7e29e089e1.slice/crio-a51fcd7aaa6833c031088dc045c500d58ce98751c027935bb41ab20f736253fa WatchSource:0}: Error finding container a51fcd7aaa6833c031088dc045c500d58ce98751c027935bb41ab20f736253fa: Status 404 returned error can't find the container with id a51fcd7aaa6833c031088dc045c500d58ce98751c027935bb41ab20f736253fa
Aug 05 17:27:01.696552 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:01.696512    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh" event={"ID":"069a1514-aa31-44e0-8192-2c7e29e089e1","Type":"ContainerStarted","Data":"a51fcd7aaa6833c031088dc045c500d58ce98751c027935bb41ab20f736253fa"}
Aug 05 17:27:03.707504 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:03.707463    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh" event={"ID":"069a1514-aa31-44e0-8192-2c7e29e089e1","Type":"ContainerStarted","Data":"cf4f06cbc842d209cd7d1fad8be6fba84d79645af7742309b197f0f90aad541c"}
Aug 05 17:27:03.707941 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:03.707530    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"
Aug 05 17:27:03.727342 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:03.727300    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh" podStartSLOduration=1.414661092 podStartE2EDuration="3.727284928s" totalImagesPullingTime="2.312623836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:27:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:27:03.726459034 +0000 UTC m=+437.144773791" watchObservedRunningTime="2026-08-05 17:27:03.727284928 +0000 UTC m=+437.145599685"
Aug 05 17:27:04.609110 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:04.609078    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"]
Aug 05 17:27:04.611526 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:04.611460    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"
Aug 05 17:27:04.614361 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:04.614333    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nh2zq\""
Aug 05 17:27:04.627936 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:04.627901    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"]
Aug 05 17:27:04.713363 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:04.713330    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mkqp\" (UniqueName: \"kubernetes.io/projected/c6e6afa2-02bb-444e-9f5f-55328c0fb656-kube-api-access-8mkqp\") pod \"kuadrant-operator-controller-manager-58fc56757c-t52pw\" (UID: \"c6e6afa2-02bb-444e-9f5f-55328c0fb656\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"
Aug 05 17:27:04.713724 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:04.713385    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6e6afa2-02bb-444e-9f5f-55328c0fb656-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-t52pw\" (UID: \"c6e6afa2-02bb-444e-9f5f-55328c0fb656\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"
Aug 05 17:27:04.814411 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:04.814379    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mkqp\" (UniqueName: \"kubernetes.io/projected/c6e6afa2-02bb-444e-9f5f-55328c0fb656-kube-api-access-8mkqp\") pod \"kuadrant-operator-controller-manager-58fc56757c-t52pw\" (UID: \"c6e6afa2-02bb-444e-9f5f-55328c0fb656\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"
Aug 05 17:27:04.814542 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:04.814438    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6e6afa2-02bb-444e-9f5f-55328c0fb656-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-t52pw\" (UID: \"c6e6afa2-02bb-444e-9f5f-55328c0fb656\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"
Aug 05 17:27:04.814840 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:04.814823    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6e6afa2-02bb-444e-9f5f-55328c0fb656-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-t52pw\" (UID: \"c6e6afa2-02bb-444e-9f5f-55328c0fb656\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"
Aug 05 17:27:04.826550 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:04.826528    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mkqp\" (UniqueName: \"kubernetes.io/projected/c6e6afa2-02bb-444e-9f5f-55328c0fb656-kube-api-access-8mkqp\") pod \"kuadrant-operator-controller-manager-58fc56757c-t52pw\" (UID: \"c6e6afa2-02bb-444e-9f5f-55328c0fb656\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"
Aug 05 17:27:04.921978 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:04.921921    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"
Aug 05 17:27:05.054929 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:05.054862    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"]
Aug 05 17:27:05.057182 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:27:05.057145    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6e6afa2_02bb_444e_9f5f_55328c0fb656.slice/crio-889556cd94d3e7bea5bd50b1e015086b225ae191fd420c659772c8275b6be531 WatchSource:0}: Error finding container 889556cd94d3e7bea5bd50b1e015086b225ae191fd420c659772c8275b6be531: Status 404 returned error can't find the container with id 889556cd94d3e7bea5bd50b1e015086b225ae191fd420c659772c8275b6be531
Aug 05 17:27:05.715582 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:05.715554    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw" event={"ID":"c6e6afa2-02bb-444e-9f5f-55328c0fb656","Type":"ContainerStarted","Data":"889556cd94d3e7bea5bd50b1e015086b225ae191fd420c659772c8275b6be531"}
Aug 05 17:27:07.279732 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:07.279703    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-rr8ht"]
Aug 05 17:27:07.282811 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:07.282791    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-rr8ht"
Aug 05 17:27:07.290101 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:07.290082    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-9bbgf\""
Aug 05 17:27:07.297044 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:07.297017    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-rr8ht"]
Aug 05 17:27:07.330441 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:07.330413    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjxnv\" (UniqueName: \"kubernetes.io/projected/578f5265-ae41-4488-aff0-b03cf4c05cbe-kube-api-access-jjxnv\") pod \"authorino-operator-68d88948d8-rr8ht\" (UID: \"578f5265-ae41-4488-aff0-b03cf4c05cbe\") " pod="kuadrant-system/authorino-operator-68d88948d8-rr8ht"
Aug 05 17:27:07.431674 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:07.431651    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjxnv\" (UniqueName: \"kubernetes.io/projected/578f5265-ae41-4488-aff0-b03cf4c05cbe-kube-api-access-jjxnv\") pod \"authorino-operator-68d88948d8-rr8ht\" (UID: \"578f5265-ae41-4488-aff0-b03cf4c05cbe\") " pod="kuadrant-system/authorino-operator-68d88948d8-rr8ht"
Aug 05 17:27:07.450948 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:07.450926    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjxnv\" (UniqueName: \"kubernetes.io/projected/578f5265-ae41-4488-aff0-b03cf4c05cbe-kube-api-access-jjxnv\") pod \"authorino-operator-68d88948d8-rr8ht\" (UID: \"578f5265-ae41-4488-aff0-b03cf4c05cbe\") " pod="kuadrant-system/authorino-operator-68d88948d8-rr8ht"
Aug 05 17:27:07.593089 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:07.593022    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-rr8ht"
Aug 05 17:27:07.723615 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:07.723576    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-rr8ht"]
Aug 05 17:27:07.725927 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:27:07.725899    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod578f5265_ae41_4488_aff0_b03cf4c05cbe.slice/crio-aa3316ddfe720300e452f95aebf3f57ea5e5e380dd8b66c10170633dbb049939 WatchSource:0}: Error finding container aa3316ddfe720300e452f95aebf3f57ea5e5e380dd8b66c10170633dbb049939: Status 404 returned error can't find the container with id aa3316ddfe720300e452f95aebf3f57ea5e5e380dd8b66c10170633dbb049939
Aug 05 17:27:08.729981 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:08.729945    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-rr8ht" event={"ID":"578f5265-ae41-4488-aff0-b03cf4c05cbe","Type":"ContainerStarted","Data":"aa3316ddfe720300e452f95aebf3f57ea5e5e380dd8b66c10170633dbb049939"}
Aug 05 17:27:11.745020 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:11.744940    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw" event={"ID":"c6e6afa2-02bb-444e-9f5f-55328c0fb656","Type":"ContainerStarted","Data":"fc25d693ef44646d0a01baf5a64d919b2186b97255e28af90d030f643e51099b"}
Aug 05 17:27:11.745385 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:11.745062    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"
Aug 05 17:27:11.778958 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:11.778888    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw" podStartSLOduration=1.373140784 podStartE2EDuration="7.778872905s" totalImagesPullingTime="6.405732121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:27:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:27:11.77599212 +0000 UTC m=+445.194306892" watchObservedRunningTime="2026-08-05 17:27:11.778872905 +0000 UTC m=+445.197187661"
Aug 05 17:27:14.712983 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:14.712952    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"
Aug 05 17:27:20.785499 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:20.785466    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-rr8ht" event={"ID":"578f5265-ae41-4488-aff0-b03cf4c05cbe","Type":"ContainerStarted","Data":"6c2c2d1222512ff65ebd430a1f7d1beda4dd1a6dd664fc25095e97cd2c8050e7"}
Aug 05 17:27:20.785844 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:20.785581    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-rr8ht"
Aug 05 17:27:20.830467 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:20.830423    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-rr8ht" podStartSLOduration=1.51711518 podStartE2EDuration="13.830411625s" totalImagesPullingTime="12.313296445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:27:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:27:20.826325558 +0000 UTC m=+454.244640315" watchObservedRunningTime="2026-08-05 17:27:20.830411625 +0000 UTC m=+454.248726382"
Aug 05 17:27:22.751430 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:22.751401    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"
Aug 05 17:27:24.961493 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:24.961464    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"]
Aug 05 17:27:24.961944 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:24.961671    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw" podUID="c6e6afa2-02bb-444e-9f5f-55328c0fb656" containerName="manager" containerID="cri-o://fc25d693ef44646d0a01baf5a64d919b2186b97255e28af90d030f643e51099b" gracePeriod=2
Aug 05 17:27:24.965880 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:24.965856    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"]
Aug 05 17:27:24.981723 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:24.981702    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"]
Aug 05 17:27:24.981882 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:24.981863    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh" podUID="069a1514-aa31-44e0-8192-2c7e29e089e1" containerName="manager" containerID="cri-o://cf4f06cbc842d209cd7d1fad8be6fba84d79645af7742309b197f0f90aad541c" gracePeriod=2
Aug 05 17:27:24.994420 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:24.994393    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"]
Aug 05 17:27:24.995925 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:24.995902    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"]
Aug 05 17:27:24.996339 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:24.996322    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6e6afa2-02bb-444e-9f5f-55328c0fb656" containerName="manager"
Aug 05 17:27:24.996388 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:24.996344    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6e6afa2-02bb-444e-9f5f-55328c0fb656" containerName="manager"
Aug 05 17:27:24.996388 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:24.996377    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="069a1514-aa31-44e0-8192-2c7e29e089e1" containerName="manager"
Aug 05 17:27:24.996388 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:24.996386    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="069a1514-aa31-44e0-8192-2c7e29e089e1" containerName="manager"
Aug 05 17:27:24.996495 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:24.996483    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="069a1514-aa31-44e0-8192-2c7e29e089e1" containerName="manager"
Aug 05 17:27:24.996526 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:24.996500    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6e6afa2-02bb-444e-9f5f-55328c0fb656" containerName="manager"
Aug 05 17:27:24.999594 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:24.999577    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"
Aug 05 17:27:25.004049 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:27:25.004011    2556 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"069a1514-aa31-44e0-8192-2c7e29e089e1\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-6c549b6d85-ws5wh\": pods \"limitador-operator-controller-manager-6c549b6d85-ws5wh\" not found" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"
Aug 05 17:27:25.006013 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:27:25.005985    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-t52pw\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="c6e6afa2-02bb-444e-9f5f-55328c0fb656" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"
Aug 05 17:27:25.008028 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:27:25.008002    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-t52pw\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="c6e6afa2-02bb-444e-9f5f-55328c0fb656" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"
Aug 05 17:27:25.032093 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.032070    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5mqs5"]
Aug 05 17:27:25.035534 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.035516    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"]
Aug 05 17:27:25.035634 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.035621    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5mqs5"
Aug 05 17:27:25.049137 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.049113    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5mqs5"]
Aug 05 17:27:25.057398 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:27:25.057372    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-ws5wh\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="069a1514-aa31-44e0-8192-2c7e29e089e1" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"
Aug 05 17:27:25.062730 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.062709    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/913bb084-a3f8-4ca3-ad0a-429eb9d1152a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-zdlrq\" (UID: \"913bb084-a3f8-4ca3-ad0a-429eb9d1152a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"
Aug 05 17:27:25.062803 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.062740    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlp6w\" (UniqueName: \"kubernetes.io/projected/913bb084-a3f8-4ca3-ad0a-429eb9d1152a-kube-api-access-vlp6w\") pod \"kuadrant-operator-controller-manager-58fc56757c-zdlrq\" (UID: \"913bb084-a3f8-4ca3-ad0a-429eb9d1152a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"
Aug 05 17:27:25.062855 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.062803    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h99w7\" (UniqueName: \"kubernetes.io/projected/af372020-540d-40d5-80c8-900b3b99b3a4-kube-api-access-h99w7\") pod \"limitador-operator-controller-manager-6c549b6d85-5mqs5\" (UID: \"af372020-540d-40d5-80c8-900b3b99b3a4\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5mqs5"
Aug 05 17:27:25.101870 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:27:25.101838    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-t52pw\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="c6e6afa2-02bb-444e-9f5f-55328c0fb656" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"
Aug 05 17:27:25.105013 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:27:25.104977    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-ws5wh\" is forbidden: User \"system:node:ip-10-0-141-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-37.ec2.internal' and this object" podUID="069a1514-aa31-44e0-8192-2c7e29e089e1" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"
Aug 05 17:27:25.164687 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.163682    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/913bb084-a3f8-4ca3-ad0a-429eb9d1152a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-zdlrq\" (UID: \"913bb084-a3f8-4ca3-ad0a-429eb9d1152a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"
Aug 05 17:27:25.164687 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.163716    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlp6w\" (UniqueName: \"kubernetes.io/projected/913bb084-a3f8-4ca3-ad0a-429eb9d1152a-kube-api-access-vlp6w\") pod \"kuadrant-operator-controller-manager-58fc56757c-zdlrq\" (UID: \"913bb084-a3f8-4ca3-ad0a-429eb9d1152a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"
Aug 05 17:27:25.164687 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.163772    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h99w7\" (UniqueName: \"kubernetes.io/projected/af372020-540d-40d5-80c8-900b3b99b3a4-kube-api-access-h99w7\") pod \"limitador-operator-controller-manager-6c549b6d85-5mqs5\" (UID: \"af372020-540d-40d5-80c8-900b3b99b3a4\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5mqs5"
Aug 05 17:27:25.164687 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.164664    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/913bb084-a3f8-4ca3-ad0a-429eb9d1152a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-zdlrq\" (UID: \"913bb084-a3f8-4ca3-ad0a-429eb9d1152a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"
Aug 05 17:27:25.177386 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.177362    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlp6w\" (UniqueName: \"kubernetes.io/projected/913bb084-a3f8-4ca3-ad0a-429eb9d1152a-kube-api-access-vlp6w\") pod \"kuadrant-operator-controller-manager-58fc56757c-zdlrq\" (UID: \"913bb084-a3f8-4ca3-ad0a-429eb9d1152a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"
Aug 05 17:27:25.178820 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.178796    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h99w7\" (UniqueName: \"kubernetes.io/projected/af372020-540d-40d5-80c8-900b3b99b3a4-kube-api-access-h99w7\") pod \"limitador-operator-controller-manager-6c549b6d85-5mqs5\" (UID: \"af372020-540d-40d5-80c8-900b3b99b3a4\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5mqs5"
Aug 05 17:27:25.208948 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.208926    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"
Aug 05 17:27:25.220908 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.220889    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"
Aug 05 17:27:25.264634 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.264604    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5g6bc\" (UniqueName: \"kubernetes.io/projected/069a1514-aa31-44e0-8192-2c7e29e089e1-kube-api-access-5g6bc\") pod \"069a1514-aa31-44e0-8192-2c7e29e089e1\" (UID: \"069a1514-aa31-44e0-8192-2c7e29e089e1\") "
Aug 05 17:27:25.264768 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.264656    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mkqp\" (UniqueName: \"kubernetes.io/projected/c6e6afa2-02bb-444e-9f5f-55328c0fb656-kube-api-access-8mkqp\") pod \"c6e6afa2-02bb-444e-9f5f-55328c0fb656\" (UID: \"c6e6afa2-02bb-444e-9f5f-55328c0fb656\") "
Aug 05 17:27:25.264768 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.264698    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6e6afa2-02bb-444e-9f5f-55328c0fb656-extensions-socket-volume\") pod \"c6e6afa2-02bb-444e-9f5f-55328c0fb656\" (UID: \"c6e6afa2-02bb-444e-9f5f-55328c0fb656\") "
Aug 05 17:27:25.265360 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.265327    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e6afa2-02bb-444e-9f5f-55328c0fb656-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c6e6afa2-02bb-444e-9f5f-55328c0fb656" (UID: "c6e6afa2-02bb-444e-9f5f-55328c0fb656"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:27:25.266733 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.266711    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/069a1514-aa31-44e0-8192-2c7e29e089e1-kube-api-access-5g6bc" (OuterVolumeSpecName: "kube-api-access-5g6bc") pod "069a1514-aa31-44e0-8192-2c7e29e089e1" (UID: "069a1514-aa31-44e0-8192-2c7e29e089e1"). InnerVolumeSpecName "kube-api-access-5g6bc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:27:25.266797 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.266741    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6e6afa2-02bb-444e-9f5f-55328c0fb656-kube-api-access-8mkqp" (OuterVolumeSpecName: "kube-api-access-8mkqp") pod "c6e6afa2-02bb-444e-9f5f-55328c0fb656" (UID: "c6e6afa2-02bb-444e-9f5f-55328c0fb656"). InnerVolumeSpecName "kube-api-access-8mkqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:27:25.365346 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.365328    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g6bc\" (UniqueName: \"kubernetes.io/projected/069a1514-aa31-44e0-8192-2c7e29e089e1-kube-api-access-5g6bc\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:27:25.365346 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.365347    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mkqp\" (UniqueName: \"kubernetes.io/projected/c6e6afa2-02bb-444e-9f5f-55328c0fb656-kube-api-access-8mkqp\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:27:25.365451 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.365357    2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6e6afa2-02bb-444e-9f5f-55328c0fb656-extensions-socket-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:27:25.391733 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.391714    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"
Aug 05 17:27:25.398547 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.398524    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5mqs5"
Aug 05 17:27:25.550793 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.550765    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"]
Aug 05 17:27:25.552274 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.552251    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5mqs5"]
Aug 05 17:27:25.552656 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:27:25.552629    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod913bb084_a3f8_4ca3_ad0a_429eb9d1152a.slice/crio-cf10ece41e47de7ca4c29dccb9dbc0b02e1e840d989d09983538902abd7f631d WatchSource:0}: Error finding container cf10ece41e47de7ca4c29dccb9dbc0b02e1e840d989d09983538902abd7f631d: Status 404 returned error can't find the container with id cf10ece41e47de7ca4c29dccb9dbc0b02e1e840d989d09983538902abd7f631d
Aug 05 17:27:25.553875 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:27:25.553851    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf372020_540d_40d5_80c8_900b3b99b3a4.slice/crio-d7813c194d316b775484912b36b91d4ea3aba1ba78f05fb96158daa678a992b5 WatchSource:0}: Error finding container d7813c194d316b775484912b36b91d4ea3aba1ba78f05fb96158daa678a992b5: Status 404 returned error can't find the container with id d7813c194d316b775484912b36b91d4ea3aba1ba78f05fb96158daa678a992b5
Aug 05 17:27:25.808687 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.808593    2556 generic.go:356] "Generic (PLEG): container finished" podID="069a1514-aa31-44e0-8192-2c7e29e089e1" containerID="cf4f06cbc842d209cd7d1fad8be6fba84d79645af7742309b197f0f90aad541c" exitCode=0
Aug 05 17:27:25.808687 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.808632    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ws5wh"
Aug 05 17:27:25.808902 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.808708    2556 scope.go:117] "RemoveContainer" containerID="cf4f06cbc842d209cd7d1fad8be6fba84d79645af7742309b197f0f90aad541c"
Aug 05 17:27:25.810537 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.810501    2556 generic.go:356] "Generic (PLEG): container finished" podID="c6e6afa2-02bb-444e-9f5f-55328c0fb656" containerID="fc25d693ef44646d0a01baf5a64d919b2186b97255e28af90d030f643e51099b" exitCode=0
Aug 05 17:27:25.810660 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.810547    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-t52pw"
Aug 05 17:27:25.813413 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.813384    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5mqs5" event={"ID":"af372020-540d-40d5-80c8-900b3b99b3a4","Type":"ContainerStarted","Data":"9b876d298b8aab969d73cd19869cc2b9cbab2db688f54a9e39038f8a0304ed28"}
Aug 05 17:27:25.813537 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.813433    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5mqs5" event={"ID":"af372020-540d-40d5-80c8-900b3b99b3a4","Type":"ContainerStarted","Data":"d7813c194d316b775484912b36b91d4ea3aba1ba78f05fb96158daa678a992b5"}
Aug 05 17:27:25.813537 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.813484    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5mqs5"
Aug 05 17:27:25.815688 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.815666    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq" event={"ID":"913bb084-a3f8-4ca3-ad0a-429eb9d1152a","Type":"ContainerStarted","Data":"496d7f01bef891cb9c004654dd04eee590cfa08797f4cef7e387d8383775fd08"}
Aug 05 17:27:25.815816 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.815713    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq" event={"ID":"913bb084-a3f8-4ca3-ad0a-429eb9d1152a","Type":"ContainerStarted","Data":"cf10ece41e47de7ca4c29dccb9dbc0b02e1e840d989d09983538902abd7f631d"}
Aug 05 17:27:25.815816 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.815745    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"
Aug 05 17:27:25.820325 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.820308    2556 scope.go:117] "RemoveContainer" containerID="cf4f06cbc842d209cd7d1fad8be6fba84d79645af7742309b197f0f90aad541c"
Aug 05 17:27:25.820591 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:27:25.820574    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf4f06cbc842d209cd7d1fad8be6fba84d79645af7742309b197f0f90aad541c\": container with ID starting with cf4f06cbc842d209cd7d1fad8be6fba84d79645af7742309b197f0f90aad541c not found: ID does not exist" containerID="cf4f06cbc842d209cd7d1fad8be6fba84d79645af7742309b197f0f90aad541c"
Aug 05 17:27:25.820656 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.820599    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf4f06cbc842d209cd7d1fad8be6fba84d79645af7742309b197f0f90aad541c"} err="failed to get container status \"cf4f06cbc842d209cd7d1fad8be6fba84d79645af7742309b197f0f90aad541c\": rpc error: code = NotFound desc = could not find container \"cf4f06cbc842d209cd7d1fad8be6fba84d79645af7742309b197f0f90aad541c\": container with ID starting with cf4f06cbc842d209cd7d1fad8be6fba84d79645af7742309b197f0f90aad541c not found: ID does not exist"
Aug 05 17:27:25.820656 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.820622    2556 scope.go:117] "RemoveContainer" containerID="fc25d693ef44646d0a01baf5a64d919b2186b97255e28af90d030f643e51099b"
Aug 05 17:27:25.828247 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.828135    2556 scope.go:117] "RemoveContainer" containerID="fc25d693ef44646d0a01baf5a64d919b2186b97255e28af90d030f643e51099b"
Aug 05 17:27:25.828419 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:27:25.828402    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc25d693ef44646d0a01baf5a64d919b2186b97255e28af90d030f643e51099b\": container with ID starting with fc25d693ef44646d0a01baf5a64d919b2186b97255e28af90d030f643e51099b not found: ID does not exist" containerID="fc25d693ef44646d0a01baf5a64d919b2186b97255e28af90d030f643e51099b"
Aug 05 17:27:25.828466 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.828429    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc25d693ef44646d0a01baf5a64d919b2186b97255e28af90d030f643e51099b"} err="failed to get container status \"fc25d693ef44646d0a01baf5a64d919b2186b97255e28af90d030f643e51099b\": rpc error: code = NotFound desc = could not find container \"fc25d693ef44646d0a01baf5a64d919b2186b97255e28af90d030f643e51099b\": container with ID starting with fc25d693ef44646d0a01baf5a64d919b2186b97255e28af90d030f643e51099b not found: ID does not exist"
Aug 05 17:27:25.873015 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.872977    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5mqs5" podStartSLOduration=1.872965251 podStartE2EDuration="1.872965251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:27:25.8386378 +0000 UTC m=+459.256952568" watchObservedRunningTime="2026-08-05 17:27:25.872965251 +0000 UTC m=+459.291280008"
Aug 05 17:27:25.873611 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:25.873579    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq" podStartSLOduration=1.873569518 podStartE2EDuration="1.873569518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:27:25.873034208 +0000 UTC m=+459.291348965" watchObservedRunningTime="2026-08-05 17:27:25.873569518 +0000 UTC m=+459.291884278"
Aug 05 17:27:26.779653 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:26.779619    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"]
Aug 05 17:27:26.784203 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:26.784186    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"
Aug 05 17:27:26.797928 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:26.797907    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"]
Aug 05 17:27:26.876822 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:26.876797    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/737fe500-3129-4ddd-aebb-2923e8ec8395-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-jpk5s\" (UID: \"737fe500-3129-4ddd-aebb-2923e8ec8395\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"
Aug 05 17:27:26.876990 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:26.876975    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd58v\" (UniqueName: \"kubernetes.io/projected/737fe500-3129-4ddd-aebb-2923e8ec8395-kube-api-access-sd58v\") pod \"kuadrant-operator-controller-manager-68cf589589-jpk5s\" (UID: \"737fe500-3129-4ddd-aebb-2923e8ec8395\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"
Aug 05 17:27:26.978109 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:26.978084    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd58v\" (UniqueName: \"kubernetes.io/projected/737fe500-3129-4ddd-aebb-2923e8ec8395-kube-api-access-sd58v\") pod \"kuadrant-operator-controller-manager-68cf589589-jpk5s\" (UID: \"737fe500-3129-4ddd-aebb-2923e8ec8395\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"
Aug 05 17:27:26.978241 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:26.978119    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/737fe500-3129-4ddd-aebb-2923e8ec8395-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-jpk5s\" (UID: \"737fe500-3129-4ddd-aebb-2923e8ec8395\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"
Aug 05 17:27:26.978460 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:26.978444    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/737fe500-3129-4ddd-aebb-2923e8ec8395-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-jpk5s\" (UID: \"737fe500-3129-4ddd-aebb-2923e8ec8395\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"
Aug 05 17:27:26.994653 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:26.994625    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd58v\" (UniqueName: \"kubernetes.io/projected/737fe500-3129-4ddd-aebb-2923e8ec8395-kube-api-access-sd58v\") pod \"kuadrant-operator-controller-manager-68cf589589-jpk5s\" (UID: \"737fe500-3129-4ddd-aebb-2923e8ec8395\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"
Aug 05 17:27:27.094271 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:27.094069    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"
Aug 05 17:27:27.100330 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:27.098324    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="069a1514-aa31-44e0-8192-2c7e29e089e1" path="/var/lib/kubelet/pods/069a1514-aa31-44e0-8192-2c7e29e089e1/volumes"
Aug 05 17:27:27.100330 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:27.098841    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6e6afa2-02bb-444e-9f5f-55328c0fb656" path="/var/lib/kubelet/pods/c6e6afa2-02bb-444e-9f5f-55328c0fb656/volumes"
Aug 05 17:27:27.228616 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:27.228591    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"]
Aug 05 17:27:27.231013 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:27:27.230988    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod737fe500_3129_4ddd_aebb_2923e8ec8395.slice/crio-d22f23c8fbde3fb4121ecfb624e3f21a5178cff971325d09e422d658a2c78035 WatchSource:0}: Error finding container d22f23c8fbde3fb4121ecfb624e3f21a5178cff971325d09e422d658a2c78035: Status 404 returned error can't find the container with id d22f23c8fbde3fb4121ecfb624e3f21a5178cff971325d09e422d658a2c78035
Aug 05 17:27:27.827975 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:27.827942    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s" event={"ID":"737fe500-3129-4ddd-aebb-2923e8ec8395","Type":"ContainerStarted","Data":"0a34d11d1c5bf4343d33f3df1399ac87472427f0c2be0bc176028e2bad5ecf64"}
Aug 05 17:27:27.828357 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:27.827991    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s" event={"ID":"737fe500-3129-4ddd-aebb-2923e8ec8395","Type":"ContainerStarted","Data":"d22f23c8fbde3fb4121ecfb624e3f21a5178cff971325d09e422d658a2c78035"}
Aug 05 17:27:27.828357 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:27.828081    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"
Aug 05 17:27:27.880004 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:27.879962    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s" podStartSLOduration=1.879947578 podStartE2EDuration="1.879947578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:27:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:27:27.879406887 +0000 UTC m=+461.297721644" watchObservedRunningTime="2026-08-05 17:27:27.879947578 +0000 UTC m=+461.298262334"
Aug 05 17:27:31.791249 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:31.791219    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-rr8ht"
Aug 05 17:27:36.825204 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:36.825174    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5mqs5"
Aug 05 17:27:36.825661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:36.825245    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"
Aug 05 17:27:38.833630 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:38.833601    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"
Aug 05 17:27:38.909112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:38.909081    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"]
Aug 05 17:27:38.909366 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:38.909322    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq" podUID="913bb084-a3f8-4ca3-ad0a-429eb9d1152a" containerName="manager" containerID="cri-o://496d7f01bef891cb9c004654dd04eee590cfa08797f4cef7e387d8383775fd08" gracePeriod=10
Aug 05 17:27:39.149165 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.149132    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"
Aug 05 17:27:39.261455 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.261424    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/913bb084-a3f8-4ca3-ad0a-429eb9d1152a-extensions-socket-volume\") pod \"913bb084-a3f8-4ca3-ad0a-429eb9d1152a\" (UID: \"913bb084-a3f8-4ca3-ad0a-429eb9d1152a\") "
Aug 05 17:27:39.261596 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.261465    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlp6w\" (UniqueName: \"kubernetes.io/projected/913bb084-a3f8-4ca3-ad0a-429eb9d1152a-kube-api-access-vlp6w\") pod \"913bb084-a3f8-4ca3-ad0a-429eb9d1152a\" (UID: \"913bb084-a3f8-4ca3-ad0a-429eb9d1152a\") "
Aug 05 17:27:39.261793 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.261769    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/913bb084-a3f8-4ca3-ad0a-429eb9d1152a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "913bb084-a3f8-4ca3-ad0a-429eb9d1152a" (UID: "913bb084-a3f8-4ca3-ad0a-429eb9d1152a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:27:39.263500 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.263472    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/913bb084-a3f8-4ca3-ad0a-429eb9d1152a-kube-api-access-vlp6w" (OuterVolumeSpecName: "kube-api-access-vlp6w") pod "913bb084-a3f8-4ca3-ad0a-429eb9d1152a" (UID: "913bb084-a3f8-4ca3-ad0a-429eb9d1152a"). InnerVolumeSpecName "kube-api-access-vlp6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:27:39.362792 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.362760    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlp6w\" (UniqueName: \"kubernetes.io/projected/913bb084-a3f8-4ca3-ad0a-429eb9d1152a-kube-api-access-vlp6w\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:27:39.362792 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.362788    2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/913bb084-a3f8-4ca3-ad0a-429eb9d1152a-extensions-socket-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:27:39.879981 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.879946    2556 generic.go:356] "Generic (PLEG): container finished" podID="913bb084-a3f8-4ca3-ad0a-429eb9d1152a" containerID="496d7f01bef891cb9c004654dd04eee590cfa08797f4cef7e387d8383775fd08" exitCode=0
Aug 05 17:27:39.880431 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.879987    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"
Aug 05 17:27:39.880431 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.880039    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq" event={"ID":"913bb084-a3f8-4ca3-ad0a-429eb9d1152a","Type":"ContainerDied","Data":"496d7f01bef891cb9c004654dd04eee590cfa08797f4cef7e387d8383775fd08"}
Aug 05 17:27:39.880431 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.880088    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq" event={"ID":"913bb084-a3f8-4ca3-ad0a-429eb9d1152a","Type":"ContainerDied","Data":"cf10ece41e47de7ca4c29dccb9dbc0b02e1e840d989d09983538902abd7f631d"}
Aug 05 17:27:39.880431 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.880110    2556 scope.go:117] "RemoveContainer" containerID="496d7f01bef891cb9c004654dd04eee590cfa08797f4cef7e387d8383775fd08"
Aug 05 17:27:39.888242 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.888227    2556 scope.go:117] "RemoveContainer" containerID="496d7f01bef891cb9c004654dd04eee590cfa08797f4cef7e387d8383775fd08"
Aug 05 17:27:39.888478 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:27:39.888460    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"496d7f01bef891cb9c004654dd04eee590cfa08797f4cef7e387d8383775fd08\": container with ID starting with 496d7f01bef891cb9c004654dd04eee590cfa08797f4cef7e387d8383775fd08 not found: ID does not exist" containerID="496d7f01bef891cb9c004654dd04eee590cfa08797f4cef7e387d8383775fd08"
Aug 05 17:27:39.888521 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.888486    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"496d7f01bef891cb9c004654dd04eee590cfa08797f4cef7e387d8383775fd08"} err="failed to get container status \"496d7f01bef891cb9c004654dd04eee590cfa08797f4cef7e387d8383775fd08\": rpc error: code = NotFound desc = could not find container \"496d7f01bef891cb9c004654dd04eee590cfa08797f4cef7e387d8383775fd08\": container with ID starting with 496d7f01bef891cb9c004654dd04eee590cfa08797f4cef7e387d8383775fd08 not found: ID does not exist"
Aug 05 17:27:39.904823 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.904802    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"]
Aug 05 17:27:39.910951 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:39.910932    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-zdlrq"]
Aug 05 17:27:41.095797 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:41.095758    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="913bb084-a3f8-4ca3-ad0a-429eb9d1152a" path="/var/lib/kubelet/pods/913bb084-a3f8-4ca3-ad0a-429eb9d1152a/volumes"
Aug 05 17:27:57.249264 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.249232    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"]
Aug 05 17:27:57.249622 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.249510    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="913bb084-a3f8-4ca3-ad0a-429eb9d1152a" containerName="manager"
Aug 05 17:27:57.249622 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.249521    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="913bb084-a3f8-4ca3-ad0a-429eb9d1152a" containerName="manager"
Aug 05 17:27:57.249622 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.249586    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="913bb084-a3f8-4ca3-ad0a-429eb9d1152a" containerName="manager"
Aug 05 17:27:57.252367 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.252349    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"
Aug 05 17:27:57.254843 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.254821    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-skc6r\""
Aug 05 17:27:57.254932 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.254827    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Aug 05 17:27:57.260785 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.260761    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"]
Aug 05 17:27:57.340031 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.340005    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"]
Aug 05 17:27:57.392296 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.392270    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e82b447-0bc5-4e51-aee6-e81470351b0e-config-file\") pod \"limitador-limitador-859c4d96d6-f6bc8\" (UID: \"2e82b447-0bc5-4e51-aee6-e81470351b0e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"
Aug 05 17:27:57.392424 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.392315    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2t4d\" (UniqueName: \"kubernetes.io/projected/2e82b447-0bc5-4e51-aee6-e81470351b0e-kube-api-access-c2t4d\") pod \"limitador-limitador-859c4d96d6-f6bc8\" (UID: \"2e82b447-0bc5-4e51-aee6-e81470351b0e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"
Aug 05 17:27:57.492965 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.492936    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e82b447-0bc5-4e51-aee6-e81470351b0e-config-file\") pod \"limitador-limitador-859c4d96d6-f6bc8\" (UID: \"2e82b447-0bc5-4e51-aee6-e81470351b0e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"
Aug 05 17:27:57.493091 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.492984    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2t4d\" (UniqueName: \"kubernetes.io/projected/2e82b447-0bc5-4e51-aee6-e81470351b0e-kube-api-access-c2t4d\") pod \"limitador-limitador-859c4d96d6-f6bc8\" (UID: \"2e82b447-0bc5-4e51-aee6-e81470351b0e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"
Aug 05 17:27:57.493576 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.493556    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e82b447-0bc5-4e51-aee6-e81470351b0e-config-file\") pod \"limitador-limitador-859c4d96d6-f6bc8\" (UID: \"2e82b447-0bc5-4e51-aee6-e81470351b0e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"
Aug 05 17:27:57.501582 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.501535    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2t4d\" (UniqueName: \"kubernetes.io/projected/2e82b447-0bc5-4e51-aee6-e81470351b0e-kube-api-access-c2t4d\") pod \"limitador-limitador-859c4d96d6-f6bc8\" (UID: \"2e82b447-0bc5-4e51-aee6-e81470351b0e\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"
Aug 05 17:27:57.562500 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.562475    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"
Aug 05 17:27:57.696761 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.696736    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"]
Aug 05 17:27:57.699123 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:27:57.699094    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e82b447_0bc5_4e51_aee6_e81470351b0e.slice/crio-9a8d25cf712470e3c6f0c7dad246440cd4d067cf7d218a9dd9254f6610c12dbb WatchSource:0}: Error finding container 9a8d25cf712470e3c6f0c7dad246440cd4d067cf7d218a9dd9254f6610c12dbb: Status 404 returned error can't find the container with id 9a8d25cf712470e3c6f0c7dad246440cd4d067cf7d218a9dd9254f6610c12dbb
Aug 05 17:27:57.958381 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:57.958309    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8" event={"ID":"2e82b447-0bc5-4e51-aee6-e81470351b0e","Type":"ContainerStarted","Data":"9a8d25cf712470e3c6f0c7dad246440cd4d067cf7d218a9dd9254f6610c12dbb"}
Aug 05 17:27:58.053931 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:58.053901    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-2vwlj"]
Aug 05 17:27:58.056582 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:58.056565    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-2vwlj"
Aug 05 17:27:58.059051 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:58.059031    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8kzqt\""
Aug 05 17:27:58.063359 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:58.063315    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-2vwlj"]
Aug 05 17:27:58.199033 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:58.198992    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2f7d\" (UniqueName: \"kubernetes.io/projected/00b0d7fe-d102-46a3-8a38-6891c37cf8b8-kube-api-access-n2f7d\") pod \"authorino-b54df94b4-2vwlj\" (UID: \"00b0d7fe-d102-46a3-8a38-6891c37cf8b8\") " pod="kuadrant-system/authorino-b54df94b4-2vwlj"
Aug 05 17:27:58.300085 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:58.300050    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2f7d\" (UniqueName: \"kubernetes.io/projected/00b0d7fe-d102-46a3-8a38-6891c37cf8b8-kube-api-access-n2f7d\") pod \"authorino-b54df94b4-2vwlj\" (UID: \"00b0d7fe-d102-46a3-8a38-6891c37cf8b8\") " pod="kuadrant-system/authorino-b54df94b4-2vwlj"
Aug 05 17:27:58.309029 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:58.308998    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2f7d\" (UniqueName: \"kubernetes.io/projected/00b0d7fe-d102-46a3-8a38-6891c37cf8b8-kube-api-access-n2f7d\") pod \"authorino-b54df94b4-2vwlj\" (UID: \"00b0d7fe-d102-46a3-8a38-6891c37cf8b8\") " pod="kuadrant-system/authorino-b54df94b4-2vwlj"
Aug 05 17:27:58.367079 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:58.367056    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-2vwlj"
Aug 05 17:27:58.487078 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:58.487053    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-2vwlj"]
Aug 05 17:27:58.489209 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:27:58.489187    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00b0d7fe_d102_46a3_8a38_6891c37cf8b8.slice/crio-a4006dec75928258c7a3d888fb44f7198f78f57845b37bc8b2c85a7d730c628c WatchSource:0}: Error finding container a4006dec75928258c7a3d888fb44f7198f78f57845b37bc8b2c85a7d730c628c: Status 404 returned error can't find the container with id a4006dec75928258c7a3d888fb44f7198f78f57845b37bc8b2c85a7d730c628c
Aug 05 17:27:58.963527 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:58.963495    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-2vwlj" event={"ID":"00b0d7fe-d102-46a3-8a38-6891c37cf8b8","Type":"ContainerStarted","Data":"a4006dec75928258c7a3d888fb44f7198f78f57845b37bc8b2c85a7d730c628c"}
Aug 05 17:27:59.970739 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:59.970691    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8" event={"ID":"2e82b447-0bc5-4e51-aee6-e81470351b0e","Type":"ContainerStarted","Data":"013e13506282d7434ea6bdd141030fc0eae6adaaab641e8339fd2ce74b87a998"}
Aug 05 17:27:59.971227 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:59.970784    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"
Aug 05 17:27:59.986453 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:27:59.986401    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8" podStartSLOduration=1.7618258409999998 podStartE2EDuration="2.986382294s" totalImagesPullingTime="1.224556453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:27:59.985850084 +0000 UTC m=+493.404164863" watchObservedRunningTime="2026-08-05 17:27:59.986382294 +0000 UTC m=+493.404697052"
Aug 05 17:28:00.977804 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:00.977768    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-2vwlj" event={"ID":"00b0d7fe-d102-46a3-8a38-6891c37cf8b8","Type":"ContainerStarted","Data":"8298153286be2f2ed194ca6275524c7ce85b2628a708cfe9e416f72d9e2e6bf7"}
Aug 05 17:28:00.992635 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:00.992589    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-2vwlj" podStartSLOduration=1.043858225 podStartE2EDuration="2.992574605s" totalImagesPullingTime="1.94871638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:27:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:28:00.991494897 +0000 UTC m=+494.409809648" watchObservedRunningTime="2026-08-05 17:28:00.992574605 +0000 UTC m=+494.410889415"
Aug 05 17:28:01.652168 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:01.652122    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-2vwlj"]
Aug 05 17:28:02.985894 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:02.985857    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-2vwlj" podUID="00b0d7fe-d102-46a3-8a38-6891c37cf8b8" containerName="authorino" containerID="cri-o://8298153286be2f2ed194ca6275524c7ce85b2628a708cfe9e416f72d9e2e6bf7" gracePeriod=30
Aug 05 17:28:03.225650 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:03.225627    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-2vwlj"
Aug 05 17:28:03.339604 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:03.339525    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2f7d\" (UniqueName: \"kubernetes.io/projected/00b0d7fe-d102-46a3-8a38-6891c37cf8b8-kube-api-access-n2f7d\") pod \"00b0d7fe-d102-46a3-8a38-6891c37cf8b8\" (UID: \"00b0d7fe-d102-46a3-8a38-6891c37cf8b8\") "
Aug 05 17:28:03.341478 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:03.341457    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00b0d7fe-d102-46a3-8a38-6891c37cf8b8-kube-api-access-n2f7d" (OuterVolumeSpecName: "kube-api-access-n2f7d") pod "00b0d7fe-d102-46a3-8a38-6891c37cf8b8" (UID: "00b0d7fe-d102-46a3-8a38-6891c37cf8b8"). InnerVolumeSpecName "kube-api-access-n2f7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:28:03.440600 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:03.440566    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2f7d\" (UniqueName: \"kubernetes.io/projected/00b0d7fe-d102-46a3-8a38-6891c37cf8b8-kube-api-access-n2f7d\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:28:03.991382 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:03.991348    2556 generic.go:356] "Generic (PLEG): container finished" podID="00b0d7fe-d102-46a3-8a38-6891c37cf8b8" containerID="8298153286be2f2ed194ca6275524c7ce85b2628a708cfe9e416f72d9e2e6bf7" exitCode=0
Aug 05 17:28:03.991771 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:03.991390    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-2vwlj"
Aug 05 17:28:03.991771 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:03.991436    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-2vwlj" event={"ID":"00b0d7fe-d102-46a3-8a38-6891c37cf8b8","Type":"ContainerDied","Data":"8298153286be2f2ed194ca6275524c7ce85b2628a708cfe9e416f72d9e2e6bf7"}
Aug 05 17:28:03.991771 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:03.991477    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-2vwlj" event={"ID":"00b0d7fe-d102-46a3-8a38-6891c37cf8b8","Type":"ContainerDied","Data":"a4006dec75928258c7a3d888fb44f7198f78f57845b37bc8b2c85a7d730c628c"}
Aug 05 17:28:03.991771 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:03.991495    2556 scope.go:117] "RemoveContainer" containerID="8298153286be2f2ed194ca6275524c7ce85b2628a708cfe9e416f72d9e2e6bf7"
Aug 05 17:28:03.999903 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:03.999884    2556 scope.go:117] "RemoveContainer" containerID="8298153286be2f2ed194ca6275524c7ce85b2628a708cfe9e416f72d9e2e6bf7"
Aug 05 17:28:04.000140 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:28:04.000119    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8298153286be2f2ed194ca6275524c7ce85b2628a708cfe9e416f72d9e2e6bf7\": container with ID starting with 8298153286be2f2ed194ca6275524c7ce85b2628a708cfe9e416f72d9e2e6bf7 not found: ID does not exist" containerID="8298153286be2f2ed194ca6275524c7ce85b2628a708cfe9e416f72d9e2e6bf7"
Aug 05 17:28:04.000258 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:04.000165    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8298153286be2f2ed194ca6275524c7ce85b2628a708cfe9e416f72d9e2e6bf7"} err="failed to get container status \"8298153286be2f2ed194ca6275524c7ce85b2628a708cfe9e416f72d9e2e6bf7\": rpc error: code = NotFound desc = could not find container \"8298153286be2f2ed194ca6275524c7ce85b2628a708cfe9e416f72d9e2e6bf7\": container with ID starting with 8298153286be2f2ed194ca6275524c7ce85b2628a708cfe9e416f72d9e2e6bf7 not found: ID does not exist"
Aug 05 17:28:04.011450 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:04.011429    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-2vwlj"]
Aug 05 17:28:04.017528 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:04.017509    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-2vwlj"]
Aug 05 17:28:05.096082 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:05.096048    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00b0d7fe-d102-46a3-8a38-6891c37cf8b8" path="/var/lib/kubelet/pods/00b0d7fe-d102-46a3-8a38-6891c37cf8b8/volumes"
Aug 05 17:28:10.654247 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:10.654217    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"]
Aug 05 17:28:10.654762 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:10.654396    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8" podUID="2e82b447-0bc5-4e51-aee6-e81470351b0e" containerName="limitador" containerID="cri-o://013e13506282d7434ea6bdd141030fc0eae6adaaab641e8339fd2ce74b87a998" gracePeriod=30
Aug 05 17:28:10.655393 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:10.655369    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"
Aug 05 17:28:11.212388 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:11.212363    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"
Aug 05 17:28:11.296467 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:11.296390    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2t4d\" (UniqueName: \"kubernetes.io/projected/2e82b447-0bc5-4e51-aee6-e81470351b0e-kube-api-access-c2t4d\") pod \"2e82b447-0bc5-4e51-aee6-e81470351b0e\" (UID: \"2e82b447-0bc5-4e51-aee6-e81470351b0e\") "
Aug 05 17:28:11.296622 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:11.296492    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e82b447-0bc5-4e51-aee6-e81470351b0e-config-file\") pod \"2e82b447-0bc5-4e51-aee6-e81470351b0e\" (UID: \"2e82b447-0bc5-4e51-aee6-e81470351b0e\") "
Aug 05 17:28:11.296859 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:11.296835    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e82b447-0bc5-4e51-aee6-e81470351b0e-config-file" (OuterVolumeSpecName: "config-file") pod "2e82b447-0bc5-4e51-aee6-e81470351b0e" (UID: "2e82b447-0bc5-4e51-aee6-e81470351b0e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:28:11.298492 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:11.298468    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e82b447-0bc5-4e51-aee6-e81470351b0e-kube-api-access-c2t4d" (OuterVolumeSpecName: "kube-api-access-c2t4d") pod "2e82b447-0bc5-4e51-aee6-e81470351b0e" (UID: "2e82b447-0bc5-4e51-aee6-e81470351b0e"). InnerVolumeSpecName "kube-api-access-c2t4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:28:11.397614 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:11.397587    2556 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e82b447-0bc5-4e51-aee6-e81470351b0e-config-file\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:28:11.397614 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:11.397610    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2t4d\" (UniqueName: \"kubernetes.io/projected/2e82b447-0bc5-4e51-aee6-e81470351b0e-kube-api-access-c2t4d\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:28:12.025108 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:12.025071    2556 generic.go:356] "Generic (PLEG): container finished" podID="2e82b447-0bc5-4e51-aee6-e81470351b0e" containerID="013e13506282d7434ea6bdd141030fc0eae6adaaab641e8339fd2ce74b87a998" exitCode=0
Aug 05 17:28:12.025499 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:12.025117    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"
Aug 05 17:28:12.025499 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:12.025172    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8" event={"ID":"2e82b447-0bc5-4e51-aee6-e81470351b0e","Type":"ContainerDied","Data":"013e13506282d7434ea6bdd141030fc0eae6adaaab641e8339fd2ce74b87a998"}
Aug 05 17:28:12.025499 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:12.025230    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-f6bc8" event={"ID":"2e82b447-0bc5-4e51-aee6-e81470351b0e","Type":"ContainerDied","Data":"9a8d25cf712470e3c6f0c7dad246440cd4d067cf7d218a9dd9254f6610c12dbb"}
Aug 05 17:28:12.025499 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:12.025252    2556 scope.go:117] "RemoveContainer" containerID="013e13506282d7434ea6bdd141030fc0eae6adaaab641e8339fd2ce74b87a998"
Aug 05 17:28:12.033376 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:12.033356    2556 scope.go:117] "RemoveContainer" containerID="013e13506282d7434ea6bdd141030fc0eae6adaaab641e8339fd2ce74b87a998"
Aug 05 17:28:12.033616 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:28:12.033595    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"013e13506282d7434ea6bdd141030fc0eae6adaaab641e8339fd2ce74b87a998\": container with ID starting with 013e13506282d7434ea6bdd141030fc0eae6adaaab641e8339fd2ce74b87a998 not found: ID does not exist" containerID="013e13506282d7434ea6bdd141030fc0eae6adaaab641e8339fd2ce74b87a998"
Aug 05 17:28:12.033689 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:12.033627    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"013e13506282d7434ea6bdd141030fc0eae6adaaab641e8339fd2ce74b87a998"} err="failed to get container status \"013e13506282d7434ea6bdd141030fc0eae6adaaab641e8339fd2ce74b87a998\": rpc error: code = NotFound desc = could not find container \"013e13506282d7434ea6bdd141030fc0eae6adaaab641e8339fd2ce74b87a998\": container with ID starting with 013e13506282d7434ea6bdd141030fc0eae6adaaab641e8339fd2ce74b87a998 not found: ID does not exist"
Aug 05 17:28:12.045549 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:12.045528    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"]
Aug 05 17:28:12.048944 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:12.048924    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-f6bc8"]
Aug 05 17:28:13.096165 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:13.096122    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e82b447-0bc5-4e51-aee6-e81470351b0e" path="/var/lib/kubelet/pods/2e82b447-0bc5-4e51-aee6-e81470351b0e/volumes"
Aug 05 17:28:13.933540 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:13.933506    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r"]
Aug 05 17:28:13.933973 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:13.933956    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00b0d7fe-d102-46a3-8a38-6891c37cf8b8" containerName="authorino"
Aug 05 17:28:13.934025 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:13.933979    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="00b0d7fe-d102-46a3-8a38-6891c37cf8b8" containerName="authorino"
Aug 05 17:28:13.934025 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:13.934015    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e82b447-0bc5-4e51-aee6-e81470351b0e" containerName="limitador"
Aug 05 17:28:13.934025 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:13.934023    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e82b447-0bc5-4e51-aee6-e81470351b0e" containerName="limitador"
Aug 05 17:28:13.934188 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:13.934110    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="00b0d7fe-d102-46a3-8a38-6891c37cf8b8" containerName="authorino"
Aug 05 17:28:13.934188 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:13.934126    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e82b447-0bc5-4e51-aee6-e81470351b0e" containerName="limitador"
Aug 05 17:28:13.938673 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:13.938653    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r"
Aug 05 17:28:13.941248 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:13.941229    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 05 17:28:13.942452 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:13.942434    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-lp72k\""
Aug 05 17:28:13.942605 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:13.942579    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Aug 05 17:28:13.942895 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:13.942871    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Aug 05 17:28:13.944212 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:13.944194    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r"]
Aug 05 17:28:14.117606 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:14.117579    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tsbf\" (UniqueName: \"kubernetes.io/projected/c72eab45-b832-4766-a2ff-d2562066b00a-kube-api-access-7tsbf\") pod \"postgres-69fb4594cf-fjh8r\" (UID: \"c72eab45-b832-4766-a2ff-d2562066b00a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r"
Aug 05 17:28:14.117944 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:14.117647    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c72eab45-b832-4766-a2ff-d2562066b00a-data\") pod \"postgres-69fb4594cf-fjh8r\" (UID: \"c72eab45-b832-4766-a2ff-d2562066b00a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r"
Aug 05 17:28:14.218943 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:14.218918    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c72eab45-b832-4766-a2ff-d2562066b00a-data\") pod \"postgres-69fb4594cf-fjh8r\" (UID: \"c72eab45-b832-4766-a2ff-d2562066b00a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r"
Aug 05 17:28:14.219082 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:14.218979    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tsbf\" (UniqueName: \"kubernetes.io/projected/c72eab45-b832-4766-a2ff-d2562066b00a-kube-api-access-7tsbf\") pod \"postgres-69fb4594cf-fjh8r\" (UID: \"c72eab45-b832-4766-a2ff-d2562066b00a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r"
Aug 05 17:28:14.219369 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:14.219352    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c72eab45-b832-4766-a2ff-d2562066b00a-data\") pod \"postgres-69fb4594cf-fjh8r\" (UID: \"c72eab45-b832-4766-a2ff-d2562066b00a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r"
Aug 05 17:28:14.227033 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:14.227008    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tsbf\" (UniqueName: \"kubernetes.io/projected/c72eab45-b832-4766-a2ff-d2562066b00a-kube-api-access-7tsbf\") pod \"postgres-69fb4594cf-fjh8r\" (UID: \"c72eab45-b832-4766-a2ff-d2562066b00a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r"
Aug 05 17:28:14.250943 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:14.250919    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r"
Aug 05 17:28:14.374288 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:14.374266    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r"]
Aug 05 17:28:14.376798 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:28:14.376770    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc72eab45_b832_4766_a2ff_d2562066b00a.slice/crio-af590d1a20e466bff0de95e4bd379dc5e09dd39791043f3f11e6810d5a215c4a WatchSource:0}: Error finding container af590d1a20e466bff0de95e4bd379dc5e09dd39791043f3f11e6810d5a215c4a: Status 404 returned error can't find the container with id af590d1a20e466bff0de95e4bd379dc5e09dd39791043f3f11e6810d5a215c4a
Aug 05 17:28:15.039907 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:15.039871    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r" event={"ID":"c72eab45-b832-4766-a2ff-d2562066b00a","Type":"ContainerStarted","Data":"af590d1a20e466bff0de95e4bd379dc5e09dd39791043f3f11e6810d5a215c4a"}
Aug 05 17:28:20.065232 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:20.065190    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r" event={"ID":"c72eab45-b832-4766-a2ff-d2562066b00a","Type":"ContainerStarted","Data":"d2f3eea56e590bbb16c221ee3a4ac6d137a06e2da78e7cff65ed1a828bcbeb9f"}
Aug 05 17:28:20.065630 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:20.065250    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r"
Aug 05 17:28:20.081454 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:20.081403    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r" podStartSLOduration=2.120281013 podStartE2EDuration="7.081388867s" totalImagesPullingTime="4.961107854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:28:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:28:20.080114712 +0000 UTC m=+513.498429489" watchObservedRunningTime="2026-08-05 17:28:20.081388867 +0000 UTC m=+513.499703687"
Aug 05 17:28:26.096624 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:26.096594    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjh8r"
Aug 05 17:28:26.774615 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:26.774584    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-687bd45585-9vqzp"]
Aug 05 17:28:26.784521 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:26.784493    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-9vqzp"
Aug 05 17:28:26.784937 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:26.784917    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-687bd45585-9vqzp"]
Aug 05 17:28:26.788360 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:26.788333    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8kzqt\""
Aug 05 17:28:26.788583 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:26.788559    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Aug 05 17:28:26.924771 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:26.924733    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aa96d22b-9f32-4a67-85f3-64d8b0020d07-tls-cert\") pod \"authorino-687bd45585-9vqzp\" (UID: \"aa96d22b-9f32-4a67-85f3-64d8b0020d07\") " pod="kuadrant-system/authorino-687bd45585-9vqzp"
Aug 05 17:28:26.924937 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:26.924821    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t6m4\" (UniqueName: \"kubernetes.io/projected/aa96d22b-9f32-4a67-85f3-64d8b0020d07-kube-api-access-7t6m4\") pod \"authorino-687bd45585-9vqzp\" (UID: \"aa96d22b-9f32-4a67-85f3-64d8b0020d07\") " pod="kuadrant-system/authorino-687bd45585-9vqzp"
Aug 05 17:28:26.969605 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:26.969573    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-9vqzp"]
Aug 05 17:28:26.969819 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:28:26.969801    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7t6m4 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-687bd45585-9vqzp" podUID="aa96d22b-9f32-4a67-85f3-64d8b0020d07"
Aug 05 17:28:27.025704 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:27.025642    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aa96d22b-9f32-4a67-85f3-64d8b0020d07-tls-cert\") pod \"authorino-687bd45585-9vqzp\" (UID: \"aa96d22b-9f32-4a67-85f3-64d8b0020d07\") " pod="kuadrant-system/authorino-687bd45585-9vqzp"
Aug 05 17:28:27.025704 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:27.025677    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t6m4\" (UniqueName: \"kubernetes.io/projected/aa96d22b-9f32-4a67-85f3-64d8b0020d07-kube-api-access-7t6m4\") pod \"authorino-687bd45585-9vqzp\" (UID: \"aa96d22b-9f32-4a67-85f3-64d8b0020d07\") " pod="kuadrant-system/authorino-687bd45585-9vqzp"
Aug 05 17:28:27.028201 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:27.028179    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aa96d22b-9f32-4a67-85f3-64d8b0020d07-tls-cert\") pod \"authorino-687bd45585-9vqzp\" (UID: \"aa96d22b-9f32-4a67-85f3-64d8b0020d07\") " pod="kuadrant-system/authorino-687bd45585-9vqzp"
Aug 05 17:28:27.034208 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:27.034186    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t6m4\" (UniqueName: \"kubernetes.io/projected/aa96d22b-9f32-4a67-85f3-64d8b0020d07-kube-api-access-7t6m4\") pod \"authorino-687bd45585-9vqzp\" (UID: \"aa96d22b-9f32-4a67-85f3-64d8b0020d07\") " pod="kuadrant-system/authorino-687bd45585-9vqzp"
Aug 05 17:28:27.094127 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:27.094100    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-9vqzp"
Aug 05 17:28:27.109883 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:27.109856    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-9vqzp"
Aug 05 17:28:27.228294 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:27.228263    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7t6m4\" (UniqueName: \"kubernetes.io/projected/aa96d22b-9f32-4a67-85f3-64d8b0020d07-kube-api-access-7t6m4\") pod \"aa96d22b-9f32-4a67-85f3-64d8b0020d07\" (UID: \"aa96d22b-9f32-4a67-85f3-64d8b0020d07\") "
Aug 05 17:28:27.228434 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:27.228355    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aa96d22b-9f32-4a67-85f3-64d8b0020d07-tls-cert\") pod \"aa96d22b-9f32-4a67-85f3-64d8b0020d07\" (UID: \"aa96d22b-9f32-4a67-85f3-64d8b0020d07\") "
Aug 05 17:28:27.230285 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:27.230265    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa96d22b-9f32-4a67-85f3-64d8b0020d07-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "aa96d22b-9f32-4a67-85f3-64d8b0020d07" (UID: "aa96d22b-9f32-4a67-85f3-64d8b0020d07"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:28:27.230401 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:27.230380    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa96d22b-9f32-4a67-85f3-64d8b0020d07-kube-api-access-7t6m4" (OuterVolumeSpecName: "kube-api-access-7t6m4") pod "aa96d22b-9f32-4a67-85f3-64d8b0020d07" (UID: "aa96d22b-9f32-4a67-85f3-64d8b0020d07"). InnerVolumeSpecName "kube-api-access-7t6m4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:28:27.328783 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:27.328724    2556 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aa96d22b-9f32-4a67-85f3-64d8b0020d07-tls-cert\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:28:27.328783 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:27.328748    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t6m4\" (UniqueName: \"kubernetes.io/projected/aa96d22b-9f32-4a67-85f3-64d8b0020d07-kube-api-access-7t6m4\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:28:28.098409 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:28.098384    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-9vqzp"
Aug 05 17:28:28.129262 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:28.129235    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-9vqzp"]
Aug 05 17:28:28.133248 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:28.133224    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-687bd45585-9vqzp"]
Aug 05 17:28:29.096331 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:29.096301    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa96d22b-9f32-4a67-85f3-64d8b0020d07" path="/var/lib/kubelet/pods/aa96d22b-9f32-4a67-85f3-64d8b0020d07/volumes"
Aug 05 17:28:35.763023 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.762993    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b999f6f6f-487pj"]
Aug 05 17:28:35.773278 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.773250    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b999f6f6f-487pj"
Aug 05 17:28:35.776667 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.776643    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rbkkv\""
Aug 05 17:28:35.776808 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.776716    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 05 17:28:35.776808 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.776654    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 05 17:28:35.777585 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.777563    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b999f6f6f-487pj"]
Aug 05 17:28:35.792530 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.792501    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f50b7e0b-2f47-408d-a117-bdb960475d9e-webhook-cert\") pod \"maas-controller-6b999f6f6f-487pj\" (UID: \"f50b7e0b-2f47-408d-a117-bdb960475d9e\") " pod="opendatahub/maas-controller-6b999f6f6f-487pj"
Aug 05 17:28:35.792631 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.792536    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6268\" (UniqueName: \"kubernetes.io/projected/f50b7e0b-2f47-408d-a117-bdb960475d9e-kube-api-access-x6268\") pod \"maas-controller-6b999f6f6f-487pj\" (UID: \"f50b7e0b-2f47-408d-a117-bdb960475d9e\") " pod="opendatahub/maas-controller-6b999f6f6f-487pj"
Aug 05 17:28:35.893015 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.892987    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f50b7e0b-2f47-408d-a117-bdb960475d9e-webhook-cert\") pod \"maas-controller-6b999f6f6f-487pj\" (UID: \"f50b7e0b-2f47-408d-a117-bdb960475d9e\") " pod="opendatahub/maas-controller-6b999f6f6f-487pj"
Aug 05 17:28:35.893136 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.893029    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6268\" (UniqueName: \"kubernetes.io/projected/f50b7e0b-2f47-408d-a117-bdb960475d9e-kube-api-access-x6268\") pod \"maas-controller-6b999f6f6f-487pj\" (UID: \"f50b7e0b-2f47-408d-a117-bdb960475d9e\") " pod="opendatahub/maas-controller-6b999f6f6f-487pj"
Aug 05 17:28:35.893136 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:28:35.893126    2556 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Aug 05 17:28:35.893283 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:28:35.893208    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f50b7e0b-2f47-408d-a117-bdb960475d9e-webhook-cert podName:f50b7e0b-2f47-408d-a117-bdb960475d9e nodeName:}" failed. No retries permitted until 2026-08-05 17:28:36.393188352 +0000 UTC m=+529.811503087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/f50b7e0b-2f47-408d-a117-bdb960475d9e-webhook-cert") pod "maas-controller-6b999f6f6f-487pj" (UID: "f50b7e0b-2f47-408d-a117-bdb960475d9e") : secret "maas-controller-webhook-cert" not found
Aug 05 17:28:35.902504 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.902482    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6268\" (UniqueName: \"kubernetes.io/projected/f50b7e0b-2f47-408d-a117-bdb960475d9e-kube-api-access-x6268\") pod \"maas-controller-6b999f6f6f-487pj\" (UID: \"f50b7e0b-2f47-408d-a117-bdb960475d9e\") " pod="opendatahub/maas-controller-6b999f6f6f-487pj"
Aug 05 17:28:35.937015 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.936991    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c55ff665-k5mqh"]
Aug 05 17:28:35.939758 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.939740    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c55ff665-k5mqh"
Aug 05 17:28:35.951449 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.951427    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c55ff665-k5mqh"]
Aug 05 17:28:35.994228 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.994197    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g68ws\" (UniqueName: \"kubernetes.io/projected/499558df-d71a-43a1-8458-8063b963335d-kube-api-access-g68ws\") pod \"maas-controller-5c55ff665-k5mqh\" (UID: \"499558df-d71a-43a1-8458-8063b963335d\") " pod="opendatahub/maas-controller-5c55ff665-k5mqh"
Aug 05 17:28:35.994334 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:35.994246    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/499558df-d71a-43a1-8458-8063b963335d-webhook-cert\") pod \"maas-controller-5c55ff665-k5mqh\" (UID: \"499558df-d71a-43a1-8458-8063b963335d\") " pod="opendatahub/maas-controller-5c55ff665-k5mqh"
Aug 05 17:28:36.095018 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:36.094953    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g68ws\" (UniqueName: \"kubernetes.io/projected/499558df-d71a-43a1-8458-8063b963335d-kube-api-access-g68ws\") pod \"maas-controller-5c55ff665-k5mqh\" (UID: \"499558df-d71a-43a1-8458-8063b963335d\") " pod="opendatahub/maas-controller-5c55ff665-k5mqh"
Aug 05 17:28:36.095018 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:36.094988    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/499558df-d71a-43a1-8458-8063b963335d-webhook-cert\") pod \"maas-controller-5c55ff665-k5mqh\" (UID: \"499558df-d71a-43a1-8458-8063b963335d\") " pod="opendatahub/maas-controller-5c55ff665-k5mqh"
Aug 05 17:28:36.095209 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:28:36.095189    2556 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Aug 05 17:28:36.095281 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:28:36.095270    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/499558df-d71a-43a1-8458-8063b963335d-webhook-cert podName:499558df-d71a-43a1-8458-8063b963335d nodeName:}" failed. No retries permitted until 2026-08-05 17:28:36.59525046 +0000 UTC m=+530.013565211 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/499558df-d71a-43a1-8458-8063b963335d-webhook-cert") pod "maas-controller-5c55ff665-k5mqh" (UID: "499558df-d71a-43a1-8458-8063b963335d") : secret "maas-controller-webhook-cert" not found
Aug 05 17:28:36.103729 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:36.103702    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g68ws\" (UniqueName: \"kubernetes.io/projected/499558df-d71a-43a1-8458-8063b963335d-kube-api-access-g68ws\") pod \"maas-controller-5c55ff665-k5mqh\" (UID: \"499558df-d71a-43a1-8458-8063b963335d\") " pod="opendatahub/maas-controller-5c55ff665-k5mqh"
Aug 05 17:28:36.396702 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:36.396624    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f50b7e0b-2f47-408d-a117-bdb960475d9e-webhook-cert\") pod \"maas-controller-6b999f6f6f-487pj\" (UID: \"f50b7e0b-2f47-408d-a117-bdb960475d9e\") " pod="opendatahub/maas-controller-6b999f6f6f-487pj"
Aug 05 17:28:36.396837 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:28:36.396767    2556 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Aug 05 17:28:36.396837 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:28:36.396823    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f50b7e0b-2f47-408d-a117-bdb960475d9e-webhook-cert podName:f50b7e0b-2f47-408d-a117-bdb960475d9e nodeName:}" failed. No retries permitted until 2026-08-05 17:28:37.396807848 +0000 UTC m=+530.815122595 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/f50b7e0b-2f47-408d-a117-bdb960475d9e-webhook-cert") pod "maas-controller-6b999f6f6f-487pj" (UID: "f50b7e0b-2f47-408d-a117-bdb960475d9e") : secret "maas-controller-webhook-cert" not found
Aug 05 17:28:36.598125 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:36.598096    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/499558df-d71a-43a1-8458-8063b963335d-webhook-cert\") pod \"maas-controller-5c55ff665-k5mqh\" (UID: \"499558df-d71a-43a1-8458-8063b963335d\") " pod="opendatahub/maas-controller-5c55ff665-k5mqh"
Aug 05 17:28:36.598281 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:28:36.598262    2556 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Aug 05 17:28:36.598344 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:28:36.598333    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/499558df-d71a-43a1-8458-8063b963335d-webhook-cert podName:499558df-d71a-43a1-8458-8063b963335d nodeName:}" failed. No retries permitted until 2026-08-05 17:28:37.598312948 +0000 UTC m=+531.016627690 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/499558df-d71a-43a1-8458-8063b963335d-webhook-cert") pod "maas-controller-5c55ff665-k5mqh" (UID: "499558df-d71a-43a1-8458-8063b963335d") : secret "maas-controller-webhook-cert" not found
Aug 05 17:28:37.404060 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:37.404030    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f50b7e0b-2f47-408d-a117-bdb960475d9e-webhook-cert\") pod \"maas-controller-6b999f6f6f-487pj\" (UID: \"f50b7e0b-2f47-408d-a117-bdb960475d9e\") " pod="opendatahub/maas-controller-6b999f6f6f-487pj"
Aug 05 17:28:37.406363 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:37.406342    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f50b7e0b-2f47-408d-a117-bdb960475d9e-webhook-cert\") pod \"maas-controller-6b999f6f6f-487pj\" (UID: \"f50b7e0b-2f47-408d-a117-bdb960475d9e\") " pod="opendatahub/maas-controller-6b999f6f6f-487pj"
Aug 05 17:28:37.586753 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:37.586726    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b999f6f6f-487pj"
Aug 05 17:28:37.606507 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:37.606477    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/499558df-d71a-43a1-8458-8063b963335d-webhook-cert\") pod \"maas-controller-5c55ff665-k5mqh\" (UID: \"499558df-d71a-43a1-8458-8063b963335d\") " pod="opendatahub/maas-controller-5c55ff665-k5mqh"
Aug 05 17:28:37.608694 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:37.608668    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/499558df-d71a-43a1-8458-8063b963335d-webhook-cert\") pod \"maas-controller-5c55ff665-k5mqh\" (UID: \"499558df-d71a-43a1-8458-8063b963335d\") " pod="opendatahub/maas-controller-5c55ff665-k5mqh"
Aug 05 17:28:37.710638 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:37.710612    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b999f6f6f-487pj"]
Aug 05 17:28:37.712562 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:28:37.712526    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf50b7e0b_2f47_408d_a117_bdb960475d9e.slice/crio-6bf983269165423e9edc1f72b1f3159040530f2a113f85334e689e0ca18a5527 WatchSource:0}: Error finding container 6bf983269165423e9edc1f72b1f3159040530f2a113f85334e689e0ca18a5527: Status 404 returned error can't find the container with id 6bf983269165423e9edc1f72b1f3159040530f2a113f85334e689e0ca18a5527
Aug 05 17:28:37.749722 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:37.749698    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c55ff665-k5mqh"
Aug 05 17:28:37.875672 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:37.875611    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c55ff665-k5mqh"]
Aug 05 17:28:37.877594 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:28:37.877566    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod499558df_d71a_43a1_8458_8063b963335d.slice/crio-f833c4191292d8e055bd1811839beef7ca1ca5e6a595a70afab81299a285b0eb WatchSource:0}: Error finding container f833c4191292d8e055bd1811839beef7ca1ca5e6a595a70afab81299a285b0eb: Status 404 returned error can't find the container with id f833c4191292d8e055bd1811839beef7ca1ca5e6a595a70afab81299a285b0eb
Aug 05 17:28:38.139403 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:38.139366    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c55ff665-k5mqh" event={"ID":"499558df-d71a-43a1-8458-8063b963335d","Type":"ContainerStarted","Data":"f833c4191292d8e055bd1811839beef7ca1ca5e6a595a70afab81299a285b0eb"}
Aug 05 17:28:38.140403 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:38.140381    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b999f6f6f-487pj" event={"ID":"f50b7e0b-2f47-408d-a117-bdb960475d9e","Type":"ContainerStarted","Data":"6bf983269165423e9edc1f72b1f3159040530f2a113f85334e689e0ca18a5527"}
Aug 05 17:28:41.155621 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:41.155583    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c55ff665-k5mqh" event={"ID":"499558df-d71a-43a1-8458-8063b963335d","Type":"ContainerStarted","Data":"13b65cc7c3e078e15df13a0eaa12d1e579e429aa94e21f364e0d4f6a1b6f6427"}
Aug 05 17:28:41.156056 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:41.155762    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c55ff665-k5mqh"
Aug 05 17:28:41.157127 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:41.157101    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b999f6f6f-487pj" event={"ID":"f50b7e0b-2f47-408d-a117-bdb960475d9e","Type":"ContainerStarted","Data":"e1fb4674432f34d34b9193288a6ab849e1f12fb1909f5bb029e89cfd478113be"}
Aug 05 17:28:41.174961 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:41.174917    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c55ff665-k5mqh" podStartSLOduration=3.851080966 podStartE2EDuration="6.174904756s" totalImagesPullingTime="2.32382379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:28:41.173476708 +0000 UTC m=+534.591791493" watchObservedRunningTime="2026-08-05 17:28:41.174904756 +0000 UTC m=+534.593219491"
Aug 05 17:28:41.192699 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:41.192660    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b999f6f6f-487pj" podStartSLOduration=3.712603948 podStartE2EDuration="6.192646361s" totalImagesPullingTime="2.480042413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:28:41.1922016 +0000 UTC m=+534.610516372" watchObservedRunningTime="2026-08-05 17:28:41.192646361 +0000 UTC m=+534.610961117"
Aug 05 17:28:42.161434 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:42.161402    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b999f6f6f-487pj"
Aug 05 17:28:52.163454 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:52.163423    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c55ff665-k5mqh"
Aug 05 17:28:52.211252 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:52.211143    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b999f6f6f-487pj"]
Aug 05 17:28:52.211465 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:52.211416    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6b999f6f6f-487pj" podUID="f50b7e0b-2f47-408d-a117-bdb960475d9e" containerName="manager" containerID="cri-o://e1fb4674432f34d34b9193288a6ab849e1f12fb1909f5bb029e89cfd478113be" gracePeriod=30
Aug 05 17:28:52.213453 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:52.213353    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b999f6f6f-487pj"
Aug 05 17:28:52.443531 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:52.443509    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b999f6f6f-487pj"
Aug 05 17:28:52.520238 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:52.520214    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6268\" (UniqueName: \"kubernetes.io/projected/f50b7e0b-2f47-408d-a117-bdb960475d9e-kube-api-access-x6268\") pod \"f50b7e0b-2f47-408d-a117-bdb960475d9e\" (UID: \"f50b7e0b-2f47-408d-a117-bdb960475d9e\") "
Aug 05 17:28:52.520397 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:52.520252    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f50b7e0b-2f47-408d-a117-bdb960475d9e-webhook-cert\") pod \"f50b7e0b-2f47-408d-a117-bdb960475d9e\" (UID: \"f50b7e0b-2f47-408d-a117-bdb960475d9e\") "
Aug 05 17:28:52.522379 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:52.522345    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f50b7e0b-2f47-408d-a117-bdb960475d9e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f50b7e0b-2f47-408d-a117-bdb960475d9e" (UID: "f50b7e0b-2f47-408d-a117-bdb960475d9e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:28:52.522477 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:52.522353    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f50b7e0b-2f47-408d-a117-bdb960475d9e-kube-api-access-x6268" (OuterVolumeSpecName: "kube-api-access-x6268") pod "f50b7e0b-2f47-408d-a117-bdb960475d9e" (UID: "f50b7e0b-2f47-408d-a117-bdb960475d9e"). InnerVolumeSpecName "kube-api-access-x6268". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:28:52.620824 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:52.620796    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6268\" (UniqueName: \"kubernetes.io/projected/f50b7e0b-2f47-408d-a117-bdb960475d9e-kube-api-access-x6268\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:28:52.620824 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:52.620819    2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f50b7e0b-2f47-408d-a117-bdb960475d9e-webhook-cert\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:28:53.211049 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:53.211019    2556 generic.go:356] "Generic (PLEG): container finished" podID="f50b7e0b-2f47-408d-a117-bdb960475d9e" containerID="e1fb4674432f34d34b9193288a6ab849e1f12fb1909f5bb029e89cfd478113be" exitCode=0
Aug 05 17:28:53.211426 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:53.211079    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b999f6f6f-487pj"
Aug 05 17:28:53.211426 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:53.211114    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b999f6f6f-487pj" event={"ID":"f50b7e0b-2f47-408d-a117-bdb960475d9e","Type":"ContainerDied","Data":"e1fb4674432f34d34b9193288a6ab849e1f12fb1909f5bb029e89cfd478113be"}
Aug 05 17:28:53.211426 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:53.211176    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b999f6f6f-487pj" event={"ID":"f50b7e0b-2f47-408d-a117-bdb960475d9e","Type":"ContainerDied","Data":"6bf983269165423e9edc1f72b1f3159040530f2a113f85334e689e0ca18a5527"}
Aug 05 17:28:53.211426 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:53.211206    2556 scope.go:117] "RemoveContainer" containerID="e1fb4674432f34d34b9193288a6ab849e1f12fb1909f5bb029e89cfd478113be"
Aug 05 17:28:53.219272 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:53.219257    2556 scope.go:117] "RemoveContainer" containerID="e1fb4674432f34d34b9193288a6ab849e1f12fb1909f5bb029e89cfd478113be"
Aug 05 17:28:53.219487 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:28:53.219469    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1fb4674432f34d34b9193288a6ab849e1f12fb1909f5bb029e89cfd478113be\": container with ID starting with e1fb4674432f34d34b9193288a6ab849e1f12fb1909f5bb029e89cfd478113be not found: ID does not exist" containerID="e1fb4674432f34d34b9193288a6ab849e1f12fb1909f5bb029e89cfd478113be"
Aug 05 17:28:53.219542 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:53.219493    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1fb4674432f34d34b9193288a6ab849e1f12fb1909f5bb029e89cfd478113be"} err="failed to get container status \"e1fb4674432f34d34b9193288a6ab849e1f12fb1909f5bb029e89cfd478113be\": rpc error: code = NotFound desc = could not find container \"e1fb4674432f34d34b9193288a6ab849e1f12fb1909f5bb029e89cfd478113be\": container with ID starting with e1fb4674432f34d34b9193288a6ab849e1f12fb1909f5bb029e89cfd478113be not found: ID does not exist"
Aug 05 17:28:53.227201 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:53.227181    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b999f6f6f-487pj"]
Aug 05 17:28:53.230330 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:53.230309    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b999f6f6f-487pj"]
Aug 05 17:28:55.096123 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:28:55.096089    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f50b7e0b-2f47-408d-a117-bdb960475d9e" path="/var/lib/kubelet/pods/f50b7e0b-2f47-408d-a117-bdb960475d9e/volumes"
Aug 05 17:29:13.269393 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.269362    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"]
Aug 05 17:29:13.269842 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.269668    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f50b7e0b-2f47-408d-a117-bdb960475d9e" containerName="manager"
Aug 05 17:29:13.269842 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.269679    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f50b7e0b-2f47-408d-a117-bdb960475d9e" containerName="manager"
Aug 05 17:29:13.269842 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.269734    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f50b7e0b-2f47-408d-a117-bdb960475d9e" containerName="manager"
Aug 05 17:29:13.272622 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.272606    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"
Aug 05 17:29:13.274915 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.274893    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Aug 05 17:29:13.283088 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.283067    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"]
Aug 05 17:29:13.374472 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.374436    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7pws\" (UniqueName: \"kubernetes.io/projected/f8c8e850-d79f-4c86-917e-0b8bf074e986-kube-api-access-r7pws\") pod \"maas-api-6475fbf58c-jt7dw\" (UID: \"f8c8e850-d79f-4c86-917e-0b8bf074e986\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"
Aug 05 17:29:13.374621 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.374487    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c8e850-d79f-4c86-917e-0b8bf074e986-maas-api-tls\") pod \"maas-api-6475fbf58c-jt7dw\" (UID: \"f8c8e850-d79f-4c86-917e-0b8bf074e986\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"
Aug 05 17:29:13.374621 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.374570    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c8e850-d79f-4c86-917e-0b8bf074e986-openshift-service-ca\") pod \"maas-api-6475fbf58c-jt7dw\" (UID: \"f8c8e850-d79f-4c86-917e-0b8bf074e986\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"
Aug 05 17:29:13.475461 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.475427    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c8e850-d79f-4c86-917e-0b8bf074e986-openshift-service-ca\") pod \"maas-api-6475fbf58c-jt7dw\" (UID: \"f8c8e850-d79f-4c86-917e-0b8bf074e986\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"
Aug 05 17:29:13.475571 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.475490    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7pws\" (UniqueName: \"kubernetes.io/projected/f8c8e850-d79f-4c86-917e-0b8bf074e986-kube-api-access-r7pws\") pod \"maas-api-6475fbf58c-jt7dw\" (UID: \"f8c8e850-d79f-4c86-917e-0b8bf074e986\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"
Aug 05 17:29:13.475663 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.475645    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c8e850-d79f-4c86-917e-0b8bf074e986-maas-api-tls\") pod \"maas-api-6475fbf58c-jt7dw\" (UID: \"f8c8e850-d79f-4c86-917e-0b8bf074e986\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"
Aug 05 17:29:13.475783 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:29:13.475766    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Aug 05 17:29:13.475839 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:29:13.475829    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8c8e850-d79f-4c86-917e-0b8bf074e986-maas-api-tls podName:f8c8e850-d79f-4c86-917e-0b8bf074e986 nodeName:}" failed. No retries permitted until 2026-08-05 17:29:13.975816063 +0000 UTC m=+567.394130802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f8c8e850-d79f-4c86-917e-0b8bf074e986-maas-api-tls") pod "maas-api-6475fbf58c-jt7dw" (UID: "f8c8e850-d79f-4c86-917e-0b8bf074e986") : secret "maas-api-serving-cert" not found
Aug 05 17:29:13.476046 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.476030    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c8e850-d79f-4c86-917e-0b8bf074e986-openshift-service-ca\") pod \"maas-api-6475fbf58c-jt7dw\" (UID: \"f8c8e850-d79f-4c86-917e-0b8bf074e986\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"
Aug 05 17:29:13.484114 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.484088    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7pws\" (UniqueName: \"kubernetes.io/projected/f8c8e850-d79f-4c86-917e-0b8bf074e986-kube-api-access-r7pws\") pod \"maas-api-6475fbf58c-jt7dw\" (UID: \"f8c8e850-d79f-4c86-917e-0b8bf074e986\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"
Aug 05 17:29:13.793607 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.793576    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx"]
Aug 05 17:29:13.812534 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.812493    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx"]
Aug 05 17:29:13.812727 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.812597    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx"
Aug 05 17:29:13.816796 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.816771    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Aug 05 17:29:13.817003 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.816839    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9pvcw\""
Aug 05 17:29:13.834950 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.834927    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-w7ssw"]
Aug 05 17:29:13.846474 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.846403    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-w7ssw"
Aug 05 17:29:13.848513 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.848493    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-w7ssw"]
Aug 05 17:29:13.979754 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.979723    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2srwq\" (UniqueName: \"kubernetes.io/projected/0244cf89-0bf2-48ba-bb1c-8e9980a77ff6-kube-api-access-2srwq\") pod \"payload-pre-processing-687dbb6cf6-z9rmx\" (UID: \"0244cf89-0bf2-48ba-bb1c-8e9980a77ff6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx"
Aug 05 17:29:13.979915 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.979773    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0244cf89-0bf2-48ba-bb1c-8e9980a77ff6-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-z9rmx\" (UID: \"0244cf89-0bf2-48ba-bb1c-8e9980a77ff6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx"
Aug 05 17:29:13.979915 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.979795    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx49f\" (UniqueName: \"kubernetes.io/projected/522e9cc8-a0a6-45f6-896a-1b893541b01e-kube-api-access-rx49f\") pod \"payload-processing-5cc6cd444b-w7ssw\" (UID: \"522e9cc8-a0a6-45f6-896a-1b893541b01e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w7ssw"
Aug 05 17:29:13.979915 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.979857    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c8e850-d79f-4c86-917e-0b8bf074e986-maas-api-tls\") pod \"maas-api-6475fbf58c-jt7dw\" (UID: \"f8c8e850-d79f-4c86-917e-0b8bf074e986\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"
Aug 05 17:29:13.979915 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.979880    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/522e9cc8-a0a6-45f6-896a-1b893541b01e-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-w7ssw\" (UID: \"522e9cc8-a0a6-45f6-896a-1b893541b01e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w7ssw"
Aug 05 17:29:13.982177 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:13.982129    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8c8e850-d79f-4c86-917e-0b8bf074e986-maas-api-tls\") pod \"maas-api-6475fbf58c-jt7dw\" (UID: \"f8c8e850-d79f-4c86-917e-0b8bf074e986\") " pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"
Aug 05 17:29:14.081267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:14.081201    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2srwq\" (UniqueName: \"kubernetes.io/projected/0244cf89-0bf2-48ba-bb1c-8e9980a77ff6-kube-api-access-2srwq\") pod \"payload-pre-processing-687dbb6cf6-z9rmx\" (UID: \"0244cf89-0bf2-48ba-bb1c-8e9980a77ff6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx"
Aug 05 17:29:14.081267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:14.081250    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0244cf89-0bf2-48ba-bb1c-8e9980a77ff6-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-z9rmx\" (UID: \"0244cf89-0bf2-48ba-bb1c-8e9980a77ff6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx"
Aug 05 17:29:14.081267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:14.081269    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx49f\" (UniqueName: \"kubernetes.io/projected/522e9cc8-a0a6-45f6-896a-1b893541b01e-kube-api-access-rx49f\") pod \"payload-processing-5cc6cd444b-w7ssw\" (UID: \"522e9cc8-a0a6-45f6-896a-1b893541b01e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w7ssw"
Aug 05 17:29:14.081464 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:14.081286    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/522e9cc8-a0a6-45f6-896a-1b893541b01e-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-w7ssw\" (UID: \"522e9cc8-a0a6-45f6-896a-1b893541b01e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w7ssw"
Aug 05 17:29:14.081929 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:14.081911    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/522e9cc8-a0a6-45f6-896a-1b893541b01e-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-w7ssw\" (UID: \"522e9cc8-a0a6-45f6-896a-1b893541b01e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w7ssw"
Aug 05 17:29:14.081971 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:14.081950    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0244cf89-0bf2-48ba-bb1c-8e9980a77ff6-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-z9rmx\" (UID: \"0244cf89-0bf2-48ba-bb1c-8e9980a77ff6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx"
Aug 05 17:29:14.089295 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:14.089270    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx49f\" (UniqueName: \"kubernetes.io/projected/522e9cc8-a0a6-45f6-896a-1b893541b01e-kube-api-access-rx49f\") pod \"payload-processing-5cc6cd444b-w7ssw\" (UID: \"522e9cc8-a0a6-45f6-896a-1b893541b01e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-w7ssw"
Aug 05 17:29:14.089390 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:14.089347    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2srwq\" (UniqueName: \"kubernetes.io/projected/0244cf89-0bf2-48ba-bb1c-8e9980a77ff6-kube-api-access-2srwq\") pod \"payload-pre-processing-687dbb6cf6-z9rmx\" (UID: \"0244cf89-0bf2-48ba-bb1c-8e9980a77ff6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx"
Aug 05 17:29:14.124833 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:14.124805    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx"
Aug 05 17:29:14.158475 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:14.158447    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-w7ssw"
Aug 05 17:29:14.184316 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:14.184287    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"
Aug 05 17:29:14.288412 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:14.288373    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx"]
Aug 05 17:29:14.291173 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:29:14.291097    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0244cf89_0bf2_48ba_bb1c_8e9980a77ff6.slice/crio-855838309bf0b141591b03bd6a8e589a82a829edaa42d10c4ed59ec1ad64ec33 WatchSource:0}: Error finding container 855838309bf0b141591b03bd6a8e589a82a829edaa42d10c4ed59ec1ad64ec33: Status 404 returned error can't find the container with id 855838309bf0b141591b03bd6a8e589a82a829edaa42d10c4ed59ec1ad64ec33
Aug 05 17:29:14.301595 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:14.301571    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx" event={"ID":"0244cf89-0bf2-48ba-bb1c-8e9980a77ff6","Type":"ContainerStarted","Data":"855838309bf0b141591b03bd6a8e589a82a829edaa42d10c4ed59ec1ad64ec33"}
Aug 05 17:29:14.336777 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:14.336525    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-w7ssw"]
Aug 05 17:29:14.338869 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:29:14.338839    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod522e9cc8_a0a6_45f6_896a_1b893541b01e.slice/crio-bb8ebd6d71c23249cf020f5fb7501a63160b3e88768b08d9232a19d28ba548fd WatchSource:0}: Error finding container bb8ebd6d71c23249cf020f5fb7501a63160b3e88768b08d9232a19d28ba548fd: Status 404 returned error can't find the container with id bb8ebd6d71c23249cf020f5fb7501a63160b3e88768b08d9232a19d28ba548fd
Aug 05 17:29:14.362832 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:14.362810    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"]
Aug 05 17:29:14.364540 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:29:14.364516    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c8e850_d79f_4c86_917e_0b8bf074e986.slice/crio-940d04be8bb2a53b9634ac73cf933b10fbdd54701f443b4eb145d912829498a0 WatchSource:0}: Error finding container 940d04be8bb2a53b9634ac73cf933b10fbdd54701f443b4eb145d912829498a0: Status 404 returned error can't find the container with id 940d04be8bb2a53b9634ac73cf933b10fbdd54701f443b4eb145d912829498a0
Aug 05 17:29:15.312397 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:15.312309    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-w7ssw" event={"ID":"522e9cc8-a0a6-45f6-896a-1b893541b01e","Type":"ContainerStarted","Data":"bb8ebd6d71c23249cf020f5fb7501a63160b3e88768b08d9232a19d28ba548fd"}
Aug 05 17:29:15.318199 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:15.318141    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw" event={"ID":"f8c8e850-d79f-4c86-917e-0b8bf074e986","Type":"ContainerStarted","Data":"940d04be8bb2a53b9634ac73cf933b10fbdd54701f443b4eb145d912829498a0"}
Aug 05 17:29:17.329357 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:17.329315    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-w7ssw" event={"ID":"522e9cc8-a0a6-45f6-896a-1b893541b01e","Type":"ContainerStarted","Data":"177d0ace461a57830082fd5c6e34566fc338edbcd7b648019ff961a901fe5392"}
Aug 05 17:29:17.329798 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:17.329392    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-w7ssw"
Aug 05 17:29:17.330775 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:17.330751    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx" event={"ID":"0244cf89-0bf2-48ba-bb1c-8e9980a77ff6","Type":"ContainerStarted","Data":"65efcb25efb9fcdd7e539bb382142537afd0e4b52dde8113a0b22c9a1c57123f"}
Aug 05 17:29:17.330869 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:17.330844    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx"
Aug 05 17:29:17.332024 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:17.332000    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw" event={"ID":"f8c8e850-d79f-4c86-917e-0b8bf074e986","Type":"ContainerStarted","Data":"da359ce0cac53c68fda8fcc57b72b01b8a4887798dda461391bce473be2a8002"}
Aug 05 17:29:17.332132 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:17.332120    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"
Aug 05 17:29:17.345971 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:17.345922    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-w7ssw" podStartSLOduration=2.203196142 podStartE2EDuration="4.345910581s" totalImagesPullingTime="2.142714439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:29:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:29:17.345730607 +0000 UTC m=+570.764045385" watchObservedRunningTime="2026-08-05 17:29:17.345910581 +0000 UTC m=+570.764225316"
Aug 05 17:29:17.364054 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:17.364017    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw" podStartSLOduration=2.255274397 podStartE2EDuration="4.364005351s" totalImagesPullingTime="2.108730954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:29:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:29:17.362609372 +0000 UTC m=+570.780924151" watchObservedRunningTime="2026-08-05 17:29:17.364005351 +0000 UTC m=+570.782320107"
Aug 05 17:29:17.379344 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:17.379308    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx" podStartSLOduration=2.197750845 podStartE2EDuration="4.379297531s" totalImagesPullingTime="2.181546686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:29:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:29:17.37868876 +0000 UTC m=+570.797003547" watchObservedRunningTime="2026-08-05 17:29:17.379297531 +0000 UTC m=+570.797612289"
Aug 05 17:29:23.341669 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:23.341635    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6475fbf58c-jt7dw"
Aug 05 17:29:28.339069 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:28.339039    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-z9rmx"
Aug 05 17:29:28.339481 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:28.339112    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-w7ssw"
Aug 05 17:29:44.270964 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.270930    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"]
Aug 05 17:29:44.273333 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.273311    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.277390 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.277368    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Aug 05 17:29:44.277487 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.277436    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-v5llc\""
Aug 05 17:29:44.277487 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.277374    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Aug 05 17:29:44.277579 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.277486    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Aug 05 17:29:44.284646 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.284625    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"]
Aug 05 17:29:44.312856 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.312828    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de8ba1b9-6b73-4600-8328-8df55c054521-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.312959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.312868    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de8ba1b9-6b73-4600-8328-8df55c054521-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.312959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.312882    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de8ba1b9-6b73-4600-8328-8df55c054521-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.312959 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.312939    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de8ba1b9-6b73-4600-8328-8df55c054521-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.313077 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.312975    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfjpm\" (UniqueName: \"kubernetes.io/projected/de8ba1b9-6b73-4600-8328-8df55c054521-kube-api-access-jfjpm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.313077 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.313001    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de8ba1b9-6b73-4600-8328-8df55c054521-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.414414 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.414383    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de8ba1b9-6b73-4600-8328-8df55c054521-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.414414 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.414413    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de8ba1b9-6b73-4600-8328-8df55c054521-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.414588 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.414545    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de8ba1b9-6b73-4600-8328-8df55c054521-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.414643 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.414595    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfjpm\" (UniqueName: \"kubernetes.io/projected/de8ba1b9-6b73-4600-8328-8df55c054521-kube-api-access-jfjpm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.414697 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.414651    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de8ba1b9-6b73-4600-8328-8df55c054521-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.414759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.414709    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de8ba1b9-6b73-4600-8328-8df55c054521-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.414814 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.414794    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de8ba1b9-6b73-4600-8328-8df55c054521-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.414918 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.414899    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de8ba1b9-6b73-4600-8328-8df55c054521-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.415027 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.415005    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de8ba1b9-6b73-4600-8328-8df55c054521-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.416586 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.416565    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de8ba1b9-6b73-4600-8328-8df55c054521-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.417207 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.417191    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de8ba1b9-6b73-4600-8328-8df55c054521-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.423225 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.423208    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfjpm\" (UniqueName: \"kubernetes.io/projected/de8ba1b9-6b73-4600-8328-8df55c054521-kube-api-access-jfjpm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g2x5t\" (UID: \"de8ba1b9-6b73-4600-8328-8df55c054521\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.583112 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.583045    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:44.715832 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:44.715800    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"]
Aug 05 17:29:44.719205 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:29:44.719177    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde8ba1b9_6b73_4600_8328_8df55c054521.slice/crio-b013152322841657065b69a82dd7c91b2831c4b459b1c4ff2a3cdc7388ba6486 WatchSource:0}: Error finding container b013152322841657065b69a82dd7c91b2831c4b459b1c4ff2a3cdc7388ba6486: Status 404 returned error can't find the container with id b013152322841657065b69a82dd7c91b2831c4b459b1c4ff2a3cdc7388ba6486
Aug 05 17:29:45.458452 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:45.458417    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t" event={"ID":"de8ba1b9-6b73-4600-8328-8df55c054521","Type":"ContainerStarted","Data":"b013152322841657065b69a82dd7c91b2831c4b459b1c4ff2a3cdc7388ba6486"}
Aug 05 17:29:46.464232 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:46.464192    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t" event={"ID":"de8ba1b9-6b73-4600-8328-8df55c054521","Type":"ContainerStarted","Data":"89631f86b61db6f744e34f05f044f2aa7c06bbd5d3b0850ad18d0f8ab9640a0b"}
Aug 05 17:29:46.485258 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:46.485214    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t" podStartSLOduration=1.528939881 podStartE2EDuration="2.485198617s" totalImagesPullingTime="956.258736ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:29:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:29:46.484393731 +0000 UTC m=+599.902708509" watchObservedRunningTime="2026-08-05 17:29:46.485198617 +0000 UTC m=+599.903513373"
Aug 05 17:29:54.583266 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:54.583239    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:54.583693 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:54.583382    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:54.595835 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:54.595813    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:29:55.513128 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:29:55.513104    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g2x5t"
Aug 05 17:30:11.654935 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.654849    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"]
Aug 05 17:30:11.657973 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.657950    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.660490 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.660470    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Aug 05 17:30:11.667295 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.667272    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"]
Aug 05 17:30:11.730121 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.730089    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c45a533-b352-4c58-b99a-2c91cb3bb618-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.730275 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.730126    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c45a533-b352-4c58-b99a-2c91cb3bb618-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.730275 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.730189    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c45a533-b352-4c58-b99a-2c91cb3bb618-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.730275 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.730226    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdj7j\" (UniqueName: \"kubernetes.io/projected/1c45a533-b352-4c58-b99a-2c91cb3bb618-kube-api-access-zdj7j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.730275 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.730262    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c45a533-b352-4c58-b99a-2c91cb3bb618-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.730415 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.730280    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c45a533-b352-4c58-b99a-2c91cb3bb618-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.830773 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.830742    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c45a533-b352-4c58-b99a-2c91cb3bb618-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.830773 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.830775    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c45a533-b352-4c58-b99a-2c91cb3bb618-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.830950 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.830806    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c45a533-b352-4c58-b99a-2c91cb3bb618-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.830950 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.830826    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c45a533-b352-4c58-b99a-2c91cb3bb618-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.831023 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.830937    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c45a533-b352-4c58-b99a-2c91cb3bb618-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.831023 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.831002    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdj7j\" (UniqueName: \"kubernetes.io/projected/1c45a533-b352-4c58-b99a-2c91cb3bb618-kube-api-access-zdj7j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.831214 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.831182    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c45a533-b352-4c58-b99a-2c91cb3bb618-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.831337 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.831234    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c45a533-b352-4c58-b99a-2c91cb3bb618-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.831337 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.831274    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c45a533-b352-4c58-b99a-2c91cb3bb618-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.833028 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.833009    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c45a533-b352-4c58-b99a-2c91cb3bb618-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.833314 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.833296    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c45a533-b352-4c58-b99a-2c91cb3bb618-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.839663 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.839640    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdj7j\" (UniqueName: \"kubernetes.io/projected/1c45a533-b352-4c58-b99a-2c91cb3bb618-kube-api-access-zdj7j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ckv2c\" (UID: \"1c45a533-b352-4c58-b99a-2c91cb3bb618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:11.968656 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:11.968629    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:12.096907 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:12.096882    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"]
Aug 05 17:30:12.099369 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:30:12.099345    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c45a533_b352_4c58_b99a_2c91cb3bb618.slice/crio-3e5623d6ddcbe7a9949aceb761249f20882dc083d14aa32a397a85942e2b2d83 WatchSource:0}: Error finding container 3e5623d6ddcbe7a9949aceb761249f20882dc083d14aa32a397a85942e2b2d83: Status 404 returned error can't find the container with id 3e5623d6ddcbe7a9949aceb761249f20882dc083d14aa32a397a85942e2b2d83
Aug 05 17:30:12.100765 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:12.100749    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:30:12.574747 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:12.574671    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c" event={"ID":"1c45a533-b352-4c58-b99a-2c91cb3bb618","Type":"ContainerStarted","Data":"170af3e83bd48e055f08d368db769516df920031ac67730e62b5112f43ba8f9f"}
Aug 05 17:30:12.574747 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:12.574728    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c" event={"ID":"1c45a533-b352-4c58-b99a-2c91cb3bb618","Type":"ContainerStarted","Data":"3e5623d6ddcbe7a9949aceb761249f20882dc083d14aa32a397a85942e2b2d83"}
Aug 05 17:30:12.592576 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:12.592478    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c" podStartSLOduration=1.421921943 podStartE2EDuration="1.592458977s" totalImagesPullingTime="170.537034ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:30:12.592235157 +0000 UTC m=+626.010549928" watchObservedRunningTime="2026-08-05 17:30:12.592458977 +0000 UTC m=+626.010773735"
Aug 05 17:30:20.952235 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:20.952199    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"]
Aug 05 17:30:20.955779 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:20.955759    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:20.958297 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:20.958278    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Aug 05 17:30:20.966037 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:20.966015    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"]
Aug 05 17:30:21.107387 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.107355    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.107558 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.107396    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.107558 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.107507    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hm8v\" (UniqueName: \"kubernetes.io/projected/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-kube-api-access-7hm8v\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.107674 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.107568    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.107674 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.107648    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.107773 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.107696    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.208651 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.208573    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.208651 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.208632    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.208848 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.208684    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.208848 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.208736    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.208848 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.208784    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hm8v\" (UniqueName: \"kubernetes.io/projected/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-kube-api-access-7hm8v\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.208848 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.208828    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.209222 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.209164    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.209222 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.209191    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.209562 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.209540    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.211227 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.211203    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.211343 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.211253    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.217237 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.217211    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hm8v\" (UniqueName: \"kubernetes.io/projected/996a6bf5-1fab-47f7-a31d-50ab1c2639fb-kube-api-access-7hm8v\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z\" (UID: \"996a6bf5-1fab-47f7-a31d-50ab1c2639fb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.265614 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.265585    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:21.402493 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.402470    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"]
Aug 05 17:30:21.404629 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:30:21.404602    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod996a6bf5_1fab_47f7_a31d_50ab1c2639fb.slice/crio-136439d04d51ca770db12653a3ac227a65a79be521fdacd2278381b3337d35fe WatchSource:0}: Error finding container 136439d04d51ca770db12653a3ac227a65a79be521fdacd2278381b3337d35fe: Status 404 returned error can't find the container with id 136439d04d51ca770db12653a3ac227a65a79be521fdacd2278381b3337d35fe
Aug 05 17:30:21.616921 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.616886    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z" event={"ID":"996a6bf5-1fab-47f7-a31d-50ab1c2639fb","Type":"ContainerStarted","Data":"136439d04d51ca770db12653a3ac227a65a79be521fdacd2278381b3337d35fe"}
Aug 05 17:30:21.969369 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.969341    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:21.969717 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.969389    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:21.982691 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:21.982673    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:22.622922 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:22.622880    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z" event={"ID":"996a6bf5-1fab-47f7-a31d-50ab1c2639fb","Type":"ContainerStarted","Data":"28bbdba694e35fecefd1392e5844c9ffd3568e224b7069fa94075c95537cf8f7"}
Aug 05 17:30:22.634410 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:22.634378    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ckv2c"
Aug 05 17:30:22.640839 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:22.640798    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z" podStartSLOduration=2.401561245 podStartE2EDuration="2.640783918s" totalImagesPullingTime="239.222673ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:30:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:30:22.639518818 +0000 UTC m=+636.057833576" watchObservedRunningTime="2026-08-05 17:30:22.640783918 +0000 UTC m=+636.059098675"
Aug 05 17:30:31.265722 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:31.265688    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:31.266180 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:31.265740    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:31.278037 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:31.278017    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:30:31.674387 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:30:31.674303    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z"
Aug 05 17:32:24.040139 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:24.040104    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c55ff665-k5mqh"]
Aug 05 17:32:24.040624 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:24.040309    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5c55ff665-k5mqh" podUID="499558df-d71a-43a1-8458-8063b963335d" containerName="manager" containerID="cri-o://13b65cc7c3e078e15df13a0eaa12d1e579e429aa94e21f364e0d4f6a1b6f6427" gracePeriod=30
Aug 05 17:32:24.162537 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:24.162507    2556 generic.go:356] "Generic (PLEG): container finished" podID="499558df-d71a-43a1-8458-8063b963335d" containerID="13b65cc7c3e078e15df13a0eaa12d1e579e429aa94e21f364e0d4f6a1b6f6427" exitCode=0
Aug 05 17:32:24.162634 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:24.162534    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c55ff665-k5mqh" event={"ID":"499558df-d71a-43a1-8458-8063b963335d","Type":"ContainerDied","Data":"13b65cc7c3e078e15df13a0eaa12d1e579e429aa94e21f364e0d4f6a1b6f6427"}
Aug 05 17:32:24.288273 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:24.288243    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c55ff665-k5mqh"
Aug 05 17:32:24.367128 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:24.367061    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/499558df-d71a-43a1-8458-8063b963335d-webhook-cert\") pod \"499558df-d71a-43a1-8458-8063b963335d\" (UID: \"499558df-d71a-43a1-8458-8063b963335d\") "
Aug 05 17:32:24.367128 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:24.367093    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g68ws\" (UniqueName: \"kubernetes.io/projected/499558df-d71a-43a1-8458-8063b963335d-kube-api-access-g68ws\") pod \"499558df-d71a-43a1-8458-8063b963335d\" (UID: \"499558df-d71a-43a1-8458-8063b963335d\") "
Aug 05 17:32:24.368974 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:24.368940    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/499558df-d71a-43a1-8458-8063b963335d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "499558df-d71a-43a1-8458-8063b963335d" (UID: "499558df-d71a-43a1-8458-8063b963335d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:32:24.369100 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:24.368997    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/499558df-d71a-43a1-8458-8063b963335d-kube-api-access-g68ws" (OuterVolumeSpecName: "kube-api-access-g68ws") pod "499558df-d71a-43a1-8458-8063b963335d" (UID: "499558df-d71a-43a1-8458-8063b963335d"). InnerVolumeSpecName "kube-api-access-g68ws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:32:24.467812 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:24.467785    2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/499558df-d71a-43a1-8458-8063b963335d-webhook-cert\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:32:24.467812 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:24.467811    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g68ws\" (UniqueName: \"kubernetes.io/projected/499558df-d71a-43a1-8458-8063b963335d-kube-api-access-g68ws\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:32:25.174020 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:25.173981    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c55ff665-k5mqh" event={"ID":"499558df-d71a-43a1-8458-8063b963335d","Type":"ContainerDied","Data":"f833c4191292d8e055bd1811839beef7ca1ca5e6a595a70afab81299a285b0eb"}
Aug 05 17:32:25.174459 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:25.174086    2556 scope.go:117] "RemoveContainer" containerID="13b65cc7c3e078e15df13a0eaa12d1e579e429aa94e21f364e0d4f6a1b6f6427"
Aug 05 17:32:25.174459 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:25.174097    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c55ff665-k5mqh"
Aug 05 17:32:25.197574 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:25.197550    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c55ff665-k5mqh"]
Aug 05 17:32:25.201548 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:25.201528    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5c55ff665-k5mqh"]
Aug 05 17:32:27.097891 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:32:27.097860    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="499558df-d71a-43a1-8458-8063b963335d" path="/var/lib/kubelet/pods/499558df-d71a-43a1-8458-8063b963335d/volumes"
Aug 05 17:33:26.034116 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.034039    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c55ff665-42qsp"]
Aug 05 17:33:26.034563 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.034412    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="499558df-d71a-43a1-8458-8063b963335d" containerName="manager"
Aug 05 17:33:26.034563 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.034424    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="499558df-d71a-43a1-8458-8063b963335d" containerName="manager"
Aug 05 17:33:26.034563 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.034499    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="499558df-d71a-43a1-8458-8063b963335d" containerName="manager"
Aug 05 17:33:26.037541 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.037517    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c55ff665-42qsp"
Aug 05 17:33:26.040138 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.040118    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 05 17:33:26.041073 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.041051    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 05 17:33:26.041073 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.041062    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rbkkv\""
Aug 05 17:33:26.046352 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.046333    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c55ff665-42qsp"]
Aug 05 17:33:26.121751 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.121724    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4pf7\" (UniqueName: \"kubernetes.io/projected/3278603b-0bb5-4660-8095-4878d0feae6a-kube-api-access-c4pf7\") pod \"maas-controller-5c55ff665-42qsp\" (UID: \"3278603b-0bb5-4660-8095-4878d0feae6a\") " pod="opendatahub/maas-controller-5c55ff665-42qsp"
Aug 05 17:33:26.121894 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.121798    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3278603b-0bb5-4660-8095-4878d0feae6a-webhook-cert\") pod \"maas-controller-5c55ff665-42qsp\" (UID: \"3278603b-0bb5-4660-8095-4878d0feae6a\") " pod="opendatahub/maas-controller-5c55ff665-42qsp"
Aug 05 17:33:26.223029 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.222997    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3278603b-0bb5-4660-8095-4878d0feae6a-webhook-cert\") pod \"maas-controller-5c55ff665-42qsp\" (UID: \"3278603b-0bb5-4660-8095-4878d0feae6a\") " pod="opendatahub/maas-controller-5c55ff665-42qsp"
Aug 05 17:33:26.223173 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.223050    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4pf7\" (UniqueName: \"kubernetes.io/projected/3278603b-0bb5-4660-8095-4878d0feae6a-kube-api-access-c4pf7\") pod \"maas-controller-5c55ff665-42qsp\" (UID: \"3278603b-0bb5-4660-8095-4878d0feae6a\") " pod="opendatahub/maas-controller-5c55ff665-42qsp"
Aug 05 17:33:26.225327 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.225306    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3278603b-0bb5-4660-8095-4878d0feae6a-webhook-cert\") pod \"maas-controller-5c55ff665-42qsp\" (UID: \"3278603b-0bb5-4660-8095-4878d0feae6a\") " pod="opendatahub/maas-controller-5c55ff665-42qsp"
Aug 05 17:33:26.232058 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.232034    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4pf7\" (UniqueName: \"kubernetes.io/projected/3278603b-0bb5-4660-8095-4878d0feae6a-kube-api-access-c4pf7\") pod \"maas-controller-5c55ff665-42qsp\" (UID: \"3278603b-0bb5-4660-8095-4878d0feae6a\") " pod="opendatahub/maas-controller-5c55ff665-42qsp"
Aug 05 17:33:26.348515 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.348459    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c55ff665-42qsp"
Aug 05 17:33:26.474592 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:26.474567    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c55ff665-42qsp"]
Aug 05 17:33:26.476283 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:33:26.476257    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3278603b_0bb5_4660_8095_4878d0feae6a.slice/crio-793fd13594d1f4b486be70df4a2373d06c9c1e75fd7010f78106c7e9733c2c2a WatchSource:0}: Error finding container 793fd13594d1f4b486be70df4a2373d06c9c1e75fd7010f78106c7e9733c2c2a: Status 404 returned error can't find the container with id 793fd13594d1f4b486be70df4a2373d06c9c1e75fd7010f78106c7e9733c2c2a
Aug 05 17:33:27.447438 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:27.447405    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c55ff665-42qsp" event={"ID":"3278603b-0bb5-4660-8095-4878d0feae6a","Type":"ContainerStarted","Data":"7192ace6cb4d45a98f13eb3921772e737d020948b25b77ebc48752dc04d5a1b5"}
Aug 05 17:33:27.447803 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:27.447463    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c55ff665-42qsp" event={"ID":"3278603b-0bb5-4660-8095-4878d0feae6a","Type":"ContainerStarted","Data":"793fd13594d1f4b486be70df4a2373d06c9c1e75fd7010f78106c7e9733c2c2a"}
Aug 05 17:33:27.447803 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:27.447489    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c55ff665-42qsp"
Aug 05 17:33:27.466298 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:27.466254    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c55ff665-42qsp" podStartSLOduration=1.02942546 podStartE2EDuration="1.466239228s" totalImagesPullingTime="436.813768ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:27.464287693 +0000 UTC m=+820.882602450" watchObservedRunningTime="2026-08-05 17:33:27.466239228 +0000 UTC m=+820.884553985"
Aug 05 17:33:38.454582 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:38.454550    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c55ff665-42qsp"
Aug 05 17:33:43.150984 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.150956    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c55ff665-42qsp"]
Aug 05 17:33:43.151385 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.151117    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5c55ff665-42qsp" podUID="3278603b-0bb5-4660-8095-4878d0feae6a" containerName="manager" containerID="cri-o://7192ace6cb4d45a98f13eb3921772e737d020948b25b77ebc48752dc04d5a1b5" gracePeriod=30
Aug 05 17:33:43.397942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.397918    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c55ff665-42qsp"
Aug 05 17:33:43.518556 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.518526    2556 generic.go:356] "Generic (PLEG): container finished" podID="3278603b-0bb5-4660-8095-4878d0feae6a" containerID="7192ace6cb4d45a98f13eb3921772e737d020948b25b77ebc48752dc04d5a1b5" exitCode=0
Aug 05 17:33:43.518720 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.518564    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c55ff665-42qsp" event={"ID":"3278603b-0bb5-4660-8095-4878d0feae6a","Type":"ContainerDied","Data":"7192ace6cb4d45a98f13eb3921772e737d020948b25b77ebc48752dc04d5a1b5"}
Aug 05 17:33:43.518720 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.518570    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c55ff665-42qsp"
Aug 05 17:33:43.518720 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.518618    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c55ff665-42qsp" event={"ID":"3278603b-0bb5-4660-8095-4878d0feae6a","Type":"ContainerDied","Data":"793fd13594d1f4b486be70df4a2373d06c9c1e75fd7010f78106c7e9733c2c2a"}
Aug 05 17:33:43.518720 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.518648    2556 scope.go:117] "RemoveContainer" containerID="7192ace6cb4d45a98f13eb3921772e737d020948b25b77ebc48752dc04d5a1b5"
Aug 05 17:33:43.526837 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.526817    2556 scope.go:117] "RemoveContainer" containerID="7192ace6cb4d45a98f13eb3921772e737d020948b25b77ebc48752dc04d5a1b5"
Aug 05 17:33:43.527074 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:33:43.527055    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7192ace6cb4d45a98f13eb3921772e737d020948b25b77ebc48752dc04d5a1b5\": container with ID starting with 7192ace6cb4d45a98f13eb3921772e737d020948b25b77ebc48752dc04d5a1b5 not found: ID does not exist" containerID="7192ace6cb4d45a98f13eb3921772e737d020948b25b77ebc48752dc04d5a1b5"
Aug 05 17:33:43.527124 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.527084    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7192ace6cb4d45a98f13eb3921772e737d020948b25b77ebc48752dc04d5a1b5"} err="failed to get container status \"7192ace6cb4d45a98f13eb3921772e737d020948b25b77ebc48752dc04d5a1b5\": rpc error: code = NotFound desc = could not find container \"7192ace6cb4d45a98f13eb3921772e737d020948b25b77ebc48752dc04d5a1b5\": container with ID starting with 7192ace6cb4d45a98f13eb3921772e737d020948b25b77ebc48752dc04d5a1b5 not found: ID does not exist"
Aug 05 17:33:43.559367 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.559345    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3278603b-0bb5-4660-8095-4878d0feae6a-webhook-cert\") pod \"3278603b-0bb5-4660-8095-4878d0feae6a\" (UID: \"3278603b-0bb5-4660-8095-4878d0feae6a\") "
Aug 05 17:33:43.559465 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.559390    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4pf7\" (UniqueName: \"kubernetes.io/projected/3278603b-0bb5-4660-8095-4878d0feae6a-kube-api-access-c4pf7\") pod \"3278603b-0bb5-4660-8095-4878d0feae6a\" (UID: \"3278603b-0bb5-4660-8095-4878d0feae6a\") "
Aug 05 17:33:43.561219 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.561196    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3278603b-0bb5-4660-8095-4878d0feae6a-kube-api-access-c4pf7" (OuterVolumeSpecName: "kube-api-access-c4pf7") pod "3278603b-0bb5-4660-8095-4878d0feae6a" (UID: "3278603b-0bb5-4660-8095-4878d0feae6a"). InnerVolumeSpecName "kube-api-access-c4pf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:33:43.561312 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.561294    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3278603b-0bb5-4660-8095-4878d0feae6a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3278603b-0bb5-4660-8095-4878d0feae6a" (UID: "3278603b-0bb5-4660-8095-4878d0feae6a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:33:43.660016 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.659990    2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3278603b-0bb5-4660-8095-4878d0feae6a-webhook-cert\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:33:43.660016 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.660011    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4pf7\" (UniqueName: \"kubernetes.io/projected/3278603b-0bb5-4660-8095-4878d0feae6a-kube-api-access-c4pf7\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:33:43.842797 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.842770    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c55ff665-42qsp"]
Aug 05 17:33:43.849103 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:43.849081    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5c55ff665-42qsp"]
Aug 05 17:33:45.012408 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.012374    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c55ff665-tctsm"]
Aug 05 17:33:45.012747 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.012723    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3278603b-0bb5-4660-8095-4878d0feae6a" containerName="manager"
Aug 05 17:33:45.012747 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.012735    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3278603b-0bb5-4660-8095-4878d0feae6a" containerName="manager"
Aug 05 17:33:45.012846 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.012812    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3278603b-0bb5-4660-8095-4878d0feae6a" containerName="manager"
Aug 05 17:33:45.016909 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.016888    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c55ff665-tctsm"
Aug 05 17:33:45.020470 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.020446    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Aug 05 17:33:45.020564 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.020453    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rbkkv\""
Aug 05 17:33:45.020564 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.020492    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Aug 05 17:33:45.025341 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.025318    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c55ff665-tctsm"]
Aug 05 17:33:45.097221 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.097193    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3278603b-0bb5-4660-8095-4878d0feae6a" path="/var/lib/kubelet/pods/3278603b-0bb5-4660-8095-4878d0feae6a/volumes"
Aug 05 17:33:45.172504 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.172473    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gthl5\" (UniqueName: \"kubernetes.io/projected/daa1e6b3-6cd2-483a-8530-536be6b95ae3-kube-api-access-gthl5\") pod \"maas-controller-5c55ff665-tctsm\" (UID: \"daa1e6b3-6cd2-483a-8530-536be6b95ae3\") " pod="opendatahub/maas-controller-5c55ff665-tctsm"
Aug 05 17:33:45.172646 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.172513    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/daa1e6b3-6cd2-483a-8530-536be6b95ae3-webhook-cert\") pod \"maas-controller-5c55ff665-tctsm\" (UID: \"daa1e6b3-6cd2-483a-8530-536be6b95ae3\") " pod="opendatahub/maas-controller-5c55ff665-tctsm"
Aug 05 17:33:45.273898 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.273834    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gthl5\" (UniqueName: \"kubernetes.io/projected/daa1e6b3-6cd2-483a-8530-536be6b95ae3-kube-api-access-gthl5\") pod \"maas-controller-5c55ff665-tctsm\" (UID: \"daa1e6b3-6cd2-483a-8530-536be6b95ae3\") " pod="opendatahub/maas-controller-5c55ff665-tctsm"
Aug 05 17:33:45.273898 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.273868    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/daa1e6b3-6cd2-483a-8530-536be6b95ae3-webhook-cert\") pod \"maas-controller-5c55ff665-tctsm\" (UID: \"daa1e6b3-6cd2-483a-8530-536be6b95ae3\") " pod="opendatahub/maas-controller-5c55ff665-tctsm"
Aug 05 17:33:45.276271 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.276254    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/daa1e6b3-6cd2-483a-8530-536be6b95ae3-webhook-cert\") pod \"maas-controller-5c55ff665-tctsm\" (UID: \"daa1e6b3-6cd2-483a-8530-536be6b95ae3\") " pod="opendatahub/maas-controller-5c55ff665-tctsm"
Aug 05 17:33:45.283931 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.283903    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gthl5\" (UniqueName: \"kubernetes.io/projected/daa1e6b3-6cd2-483a-8530-536be6b95ae3-kube-api-access-gthl5\") pod \"maas-controller-5c55ff665-tctsm\" (UID: \"daa1e6b3-6cd2-483a-8530-536be6b95ae3\") " pod="opendatahub/maas-controller-5c55ff665-tctsm"
Aug 05 17:33:45.327861 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.327839    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c55ff665-tctsm"
Aug 05 17:33:45.454636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.454613    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c55ff665-tctsm"]
Aug 05 17:33:45.455833 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:33:45.455811    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa1e6b3_6cd2_483a_8530_536be6b95ae3.slice/crio-a5e0bfc7ea5575269b8b835f1cc2e59766c0148cb60db58d497d9d8e5301086d WatchSource:0}: Error finding container a5e0bfc7ea5575269b8b835f1cc2e59766c0148cb60db58d497d9d8e5301086d: Status 404 returned error can't find the container with id a5e0bfc7ea5575269b8b835f1cc2e59766c0148cb60db58d497d9d8e5301086d
Aug 05 17:33:45.530290 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:45.530238    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c55ff665-tctsm" event={"ID":"daa1e6b3-6cd2-483a-8530-536be6b95ae3","Type":"ContainerStarted","Data":"a5e0bfc7ea5575269b8b835f1cc2e59766c0148cb60db58d497d9d8e5301086d"}
Aug 05 17:33:46.536979 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:46.536950    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c55ff665-tctsm" event={"ID":"daa1e6b3-6cd2-483a-8530-536be6b95ae3","Type":"ContainerStarted","Data":"6341a3a2722aebd967b066634524acd616cef12167883d7bcfc671ee2df9f2ed"}
Aug 05 17:33:46.537343 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:46.537035    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c55ff665-tctsm"
Aug 05 17:33:46.566498 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:46.566456    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c55ff665-tctsm" podStartSLOduration=1.574679675 podStartE2EDuration="2.566441363s" totalImagesPullingTime="991.761688ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:33:46.563726899 +0000 UTC m=+839.982041677" watchObservedRunningTime="2026-08-05 17:33:46.566441363 +0000 UTC m=+839.984756119"
Aug 05 17:33:57.544304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:33:57.544275    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c55ff665-tctsm"
Aug 05 17:40:17.805074 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:17.805038    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"]
Aug 05 17:40:17.806025 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:17.805275    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s" podUID="737fe500-3129-4ddd-aebb-2923e8ec8395" containerName="manager" containerID="cri-o://0a34d11d1c5bf4343d33f3df1399ac87472427f0c2be0bc176028e2bad5ecf64" gracePeriod=10
Aug 05 17:40:18.051893 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.051871    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"
Aug 05 17:40:18.174955 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.174875    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sd58v\" (UniqueName: \"kubernetes.io/projected/737fe500-3129-4ddd-aebb-2923e8ec8395-kube-api-access-sd58v\") pod \"737fe500-3129-4ddd-aebb-2923e8ec8395\" (UID: \"737fe500-3129-4ddd-aebb-2923e8ec8395\") "
Aug 05 17:40:18.174955 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.174945    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/737fe500-3129-4ddd-aebb-2923e8ec8395-extensions-socket-volume\") pod \"737fe500-3129-4ddd-aebb-2923e8ec8395\" (UID: \"737fe500-3129-4ddd-aebb-2923e8ec8395\") "
Aug 05 17:40:18.175384 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.175357    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/737fe500-3129-4ddd-aebb-2923e8ec8395-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "737fe500-3129-4ddd-aebb-2923e8ec8395" (UID: "737fe500-3129-4ddd-aebb-2923e8ec8395"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:40:18.176823 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.176805    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/737fe500-3129-4ddd-aebb-2923e8ec8395-kube-api-access-sd58v" (OuterVolumeSpecName: "kube-api-access-sd58v") pod "737fe500-3129-4ddd-aebb-2923e8ec8395" (UID: "737fe500-3129-4ddd-aebb-2923e8ec8395"). InnerVolumeSpecName "kube-api-access-sd58v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:40:18.275872 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.275842    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sd58v\" (UniqueName: \"kubernetes.io/projected/737fe500-3129-4ddd-aebb-2923e8ec8395-kube-api-access-sd58v\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:40:18.275872 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.275866    2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/737fe500-3129-4ddd-aebb-2923e8ec8395-extensions-socket-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:40:18.308406 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.308381    2556 generic.go:356] "Generic (PLEG): container finished" podID="737fe500-3129-4ddd-aebb-2923e8ec8395" containerID="0a34d11d1c5bf4343d33f3df1399ac87472427f0c2be0bc176028e2bad5ecf64" exitCode=0
Aug 05 17:40:18.308515 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.308430    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s" event={"ID":"737fe500-3129-4ddd-aebb-2923e8ec8395","Type":"ContainerDied","Data":"0a34d11d1c5bf4343d33f3df1399ac87472427f0c2be0bc176028e2bad5ecf64"}
Aug 05 17:40:18.308515 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.308488    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s" event={"ID":"737fe500-3129-4ddd-aebb-2923e8ec8395","Type":"ContainerDied","Data":"d22f23c8fbde3fb4121ecfb624e3f21a5178cff971325d09e422d658a2c78035"}
Aug 05 17:40:18.308515 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.308507    2556 scope.go:117] "RemoveContainer" containerID="0a34d11d1c5bf4343d33f3df1399ac87472427f0c2be0bc176028e2bad5ecf64"
Aug 05 17:40:18.308626 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.308432    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"
Aug 05 17:40:18.317864 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.317849    2556 scope.go:117] "RemoveContainer" containerID="0a34d11d1c5bf4343d33f3df1399ac87472427f0c2be0bc176028e2bad5ecf64"
Aug 05 17:40:18.318104 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:40:18.318086    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a34d11d1c5bf4343d33f3df1399ac87472427f0c2be0bc176028e2bad5ecf64\": container with ID starting with 0a34d11d1c5bf4343d33f3df1399ac87472427f0c2be0bc176028e2bad5ecf64 not found: ID does not exist" containerID="0a34d11d1c5bf4343d33f3df1399ac87472427f0c2be0bc176028e2bad5ecf64"
Aug 05 17:40:18.318170 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.318112    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a34d11d1c5bf4343d33f3df1399ac87472427f0c2be0bc176028e2bad5ecf64"} err="failed to get container status \"0a34d11d1c5bf4343d33f3df1399ac87472427f0c2be0bc176028e2bad5ecf64\": rpc error: code = NotFound desc = could not find container \"0a34d11d1c5bf4343d33f3df1399ac87472427f0c2be0bc176028e2bad5ecf64\": container with ID starting with 0a34d11d1c5bf4343d33f3df1399ac87472427f0c2be0bc176028e2bad5ecf64 not found: ID does not exist"
Aug 05 17:40:18.329915 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.329894    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"]
Aug 05 17:40:18.335641 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:18.335621    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jpk5s"]
Aug 05 17:40:19.096239 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:40:19.096211    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="737fe500-3129-4ddd-aebb-2923e8ec8395" path="/var/lib/kubelet/pods/737fe500-3129-4ddd-aebb-2923e8ec8395/volumes"
Aug 05 17:41:18.549614 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:18.549576    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"]
Aug 05 17:41:18.552004 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:18.550077    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="737fe500-3129-4ddd-aebb-2923e8ec8395" containerName="manager"
Aug 05 17:41:18.552004 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:18.550092    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="737fe500-3129-4ddd-aebb-2923e8ec8395" containerName="manager"
Aug 05 17:41:18.552004 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:18.550185    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="737fe500-3129-4ddd-aebb-2923e8ec8395" containerName="manager"
Aug 05 17:41:18.552981 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:18.552963    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"
Aug 05 17:41:18.555544 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:18.555516    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nh2zq\""
Aug 05 17:41:18.565750 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:18.565729    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"]
Aug 05 17:41:18.645472 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:18.645447    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhb8m\" (UniqueName: \"kubernetes.io/projected/3b4ddea7-f22a-4077-b9c8-c47857ac3876-kube-api-access-vhb8m\") pod \"kuadrant-operator-controller-manager-68cf589589-fpj7w\" (UID: \"3b4ddea7-f22a-4077-b9c8-c47857ac3876\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"
Aug 05 17:41:18.645608 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:18.645477    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b4ddea7-f22a-4077-b9c8-c47857ac3876-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-fpj7w\" (UID: \"3b4ddea7-f22a-4077-b9c8-c47857ac3876\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"
Aug 05 17:41:18.745896 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:18.745869    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhb8m\" (UniqueName: \"kubernetes.io/projected/3b4ddea7-f22a-4077-b9c8-c47857ac3876-kube-api-access-vhb8m\") pod \"kuadrant-operator-controller-manager-68cf589589-fpj7w\" (UID: \"3b4ddea7-f22a-4077-b9c8-c47857ac3876\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"
Aug 05 17:41:18.745996 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:18.745899    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b4ddea7-f22a-4077-b9c8-c47857ac3876-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-fpj7w\" (UID: \"3b4ddea7-f22a-4077-b9c8-c47857ac3876\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"
Aug 05 17:41:18.746290 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:18.746274    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b4ddea7-f22a-4077-b9c8-c47857ac3876-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-fpj7w\" (UID: \"3b4ddea7-f22a-4077-b9c8-c47857ac3876\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"
Aug 05 17:41:18.755166 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:18.755126    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhb8m\" (UniqueName: \"kubernetes.io/projected/3b4ddea7-f22a-4077-b9c8-c47857ac3876-kube-api-access-vhb8m\") pod \"kuadrant-operator-controller-manager-68cf589589-fpj7w\" (UID: \"3b4ddea7-f22a-4077-b9c8-c47857ac3876\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"
Aug 05 17:41:18.863368 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:18.863318    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"
Aug 05 17:41:19.010218 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:19.010192    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"]
Aug 05 17:41:19.012009 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:41:19.011972    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b4ddea7_f22a_4077_b9c8_c47857ac3876.slice/crio-a51a2bc494c848dbb766b1f70f873ca47171b77d4c91e5c1a539a99ede01af9d WatchSource:0}: Error finding container a51a2bc494c848dbb766b1f70f873ca47171b77d4c91e5c1a539a99ede01af9d: Status 404 returned error can't find the container with id a51a2bc494c848dbb766b1f70f873ca47171b77d4c91e5c1a539a99ede01af9d
Aug 05 17:41:19.568450 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:19.568415    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w" event={"ID":"3b4ddea7-f22a-4077-b9c8-c47857ac3876","Type":"ContainerStarted","Data":"cbc757989a293d5c54fffdbb2fe401d84de031286a4662688bbbdfe6c103977d"}
Aug 05 17:41:19.568811 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:19.568465    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w" event={"ID":"3b4ddea7-f22a-4077-b9c8-c47857ac3876","Type":"ContainerStarted","Data":"a51a2bc494c848dbb766b1f70f873ca47171b77d4c91e5c1a539a99ede01af9d"}
Aug 05 17:41:19.568811 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:19.568502    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"
Aug 05 17:41:19.589991 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:19.589944    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w" podStartSLOduration=1.589931288 podStartE2EDuration="1.589931288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:41:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:41:19.587403637 +0000 UTC m=+1293.005718412" watchObservedRunningTime="2026-08-05 17:41:19.589931288 +0000 UTC m=+1293.008246046"
Aug 05 17:41:30.574845 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:41:30.574816    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"
Aug 05 17:45:13.373301 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.373262    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"]
Aug 05 17:45:13.376995 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.376948    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.379456 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.379434    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-0db46b7f-kserve-self-signed-certs\""
Aug 05 17:45:13.387266 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.387245    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"]
Aug 05 17:45:13.479075 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.479043    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-dshm\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.479239 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.479082    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-model-cache\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.479239 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.479097    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-tmp-dir\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.479239 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.479121    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv9gq\" (UniqueName: \"kubernetes.io/projected/0959a421-9458-4703-a0eb-ac946ac64e9d-kube-api-access-bv9gq\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.479239 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.479225    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0959a421-9458-4703-a0eb-ac946ac64e9d-tls-certs\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.479376 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.479270    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-home\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.580110 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.580077    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-model-cache\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.580110 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.580111    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-tmp-dir\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.580309 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.580132    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv9gq\" (UniqueName: \"kubernetes.io/projected/0959a421-9458-4703-a0eb-ac946ac64e9d-kube-api-access-bv9gq\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.580309 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.580173    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0959a421-9458-4703-a0eb-ac946ac64e9d-tls-certs\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.580309 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.580212    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-home\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.580309 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.580256    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-dshm\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.580515 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.580492    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-model-cache\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.580581 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.580555    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-tmp-dir\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.580670 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.580649    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-home\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.582483 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.582463    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-dshm\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.582668 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.582650    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0959a421-9458-4703-a0eb-ac946ac64e9d-tls-certs\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.590289 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.590268    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv9gq\" (UniqueName: \"kubernetes.io/projected/0959a421-9458-4703-a0eb-ac946ac64e9d-kube-api-access-bv9gq\") pod \"e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") " pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.687076 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.687020    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:13.828750 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:13.828717    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"]
Aug 05 17:45:13.832225 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:45:13.832197    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0959a421_9458_4703_a0eb_ac946ac64e9d.slice/crio-45ddd9d5a2bf7177e216a87895b91addd1a5b8678edf60823e017a6af25b07ae WatchSource:0}: Error finding container 45ddd9d5a2bf7177e216a87895b91addd1a5b8678edf60823e017a6af25b07ae: Status 404 returned error can't find the container with id 45ddd9d5a2bf7177e216a87895b91addd1a5b8678edf60823e017a6af25b07ae
Aug 05 17:45:14.637264 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:14.637229    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8" event={"ID":"0959a421-9458-4703-a0eb-ac946ac64e9d","Type":"ContainerStarted","Data":"07425bcb4af17dae8371eabd7fa079631afd747328fb110a4229ba5ddb15c1a9"}
Aug 05 17:45:14.637264 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:14.637268    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8" event={"ID":"0959a421-9458-4703-a0eb-ac946ac64e9d","Type":"ContainerStarted","Data":"45ddd9d5a2bf7177e216a87895b91addd1a5b8678edf60823e017a6af25b07ae"}
Aug 05 17:45:14.654477 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:14.654435    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8" podStartSLOduration=1.6544196599999998 podStartE2EDuration="1.65441966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:45:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:45:14.652948165 +0000 UTC m=+1528.071262949" watchObservedRunningTime="2026-08-05 17:45:14.65441966 +0000 UTC m=+1528.072734417"
Aug 05 17:45:23.065766 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.065729    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"]
Aug 05 17:45:23.069393 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.069368    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.071987 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.071968    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-0db46b7f-kserve-self-signed-certs\""
Aug 05 17:45:23.081917 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.081897    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"]
Aug 05 17:45:23.157290 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.157260    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g2hl\" (UniqueName: \"kubernetes.io/projected/0a65a694-e8e5-4880-8539-f3d71ce31810-kube-api-access-2g2hl\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.157419 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.157296    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-model-cache\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.157419 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.157373    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a65a694-e8e5-4880-8539-f3d71ce31810-tls-certs\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.157498 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.157446    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-dshm\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.157498 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.157474    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-home\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.157498 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.157494    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-tmp-dir\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.258595 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.258559    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a65a694-e8e5-4880-8539-f3d71ce31810-tls-certs\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.258760 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.258614    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-dshm\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.258760 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.258636    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-home\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.258760 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.258656    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-tmp-dir\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.258760 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.258675    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g2hl\" (UniqueName: \"kubernetes.io/projected/0a65a694-e8e5-4880-8539-f3d71ce31810-kube-api-access-2g2hl\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.258760 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.258700    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-model-cache\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.259183 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.259139    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-model-cache\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.259275 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.259196    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-home\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.259275 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.259226    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-tmp-dir\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.260974 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.260945    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-dshm\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.261060 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.261026    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a65a694-e8e5-4880-8539-f3d71ce31810-tls-certs\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.266759 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.266735    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g2hl\" (UniqueName: \"kubernetes.io/projected/0a65a694-e8e5-4880-8539-f3d71ce31810-kube-api-access-2g2hl\") pod \"e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") " pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.379047 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.378960    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:23.523420 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.523288    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"]
Aug 05 17:45:23.526366 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:45:23.526337    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a65a694_e8e5_4880_8539_f3d71ce31810.slice/crio-a41b59cf92de7d8fb7a6fff7bd0c90e12118fd4d267d5499ddf823eaf76e3329 WatchSource:0}: Error finding container a41b59cf92de7d8fb7a6fff7bd0c90e12118fd4d267d5499ddf823eaf76e3329: Status 404 returned error can't find the container with id a41b59cf92de7d8fb7a6fff7bd0c90e12118fd4d267d5499ddf823eaf76e3329
Aug 05 17:45:23.679388 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.679303    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m" event={"ID":"0a65a694-e8e5-4880-8539-f3d71ce31810","Type":"ContainerStarted","Data":"6998ed96aa4e18de5ebd66e6f49595af1df605568f59949d27d21b84eb804be1"}
Aug 05 17:45:23.679388 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.679348    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m" event={"ID":"0a65a694-e8e5-4880-8539-f3d71ce31810","Type":"ContainerStarted","Data":"a41b59cf92de7d8fb7a6fff7bd0c90e12118fd4d267d5499ddf823eaf76e3329"}
Aug 05 17:45:23.688097 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.688073    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:23.688229 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.688112    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:23.696245 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.696109    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m" podStartSLOduration=0.696092742 podStartE2EDuration="696.092742ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:45:23.696006776 +0000 UTC m=+1537.114321539" watchObservedRunningTime="2026-08-05 17:45:23.696092742 +0000 UTC m=+1537.114407500"
Aug 05 17:45:23.700655 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:23.700631    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:24.695057 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:24.695027    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:45:33.380068 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:33.380034    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:33.380516 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:33.380082    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:33.392683 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:33.392654    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:45:33.744243 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:45:33.744211    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:46:00.896280 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:00.896244    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"]
Aug 05 17:46:02.411338 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:02.411307    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"]
Aug 05 17:46:15.917257 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:15.917211    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8" podUID="0959a421-9458-4703-a0eb-ac946ac64e9d" containerName="main" containerID="cri-o://07425bcb4af17dae8371eabd7fa079631afd747328fb110a4229ba5ddb15c1a9" gracePeriod=45
Aug 05 17:46:16.161145 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.161121    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:46:16.285838 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.285807    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-model-cache\") pod \"0959a421-9458-4703-a0eb-ac946ac64e9d\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") "
Aug 05 17:46:16.285989 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.285849    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-tmp-dir\") pod \"0959a421-9458-4703-a0eb-ac946ac64e9d\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") "
Aug 05 17:46:16.285989 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.285877    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-home\") pod \"0959a421-9458-4703-a0eb-ac946ac64e9d\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") "
Aug 05 17:46:16.285989 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.285914    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0959a421-9458-4703-a0eb-ac946ac64e9d-tls-certs\") pod \"0959a421-9458-4703-a0eb-ac946ac64e9d\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") "
Aug 05 17:46:16.285989 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.285933    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-dshm\") pod \"0959a421-9458-4703-a0eb-ac946ac64e9d\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") "
Aug 05 17:46:16.285989 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.285982    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bv9gq\" (UniqueName: \"kubernetes.io/projected/0959a421-9458-4703-a0eb-ac946ac64e9d-kube-api-access-bv9gq\") pod \"0959a421-9458-4703-a0eb-ac946ac64e9d\" (UID: \"0959a421-9458-4703-a0eb-ac946ac64e9d\") "
Aug 05 17:46:16.286253 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.286098    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0959a421-9458-4703-a0eb-ac946ac64e9d" (UID: "0959a421-9458-4703-a0eb-ac946ac64e9d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:46:16.286253 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.286105    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-model-cache" (OuterVolumeSpecName: "model-cache") pod "0959a421-9458-4703-a0eb-ac946ac64e9d" (UID: "0959a421-9458-4703-a0eb-ac946ac64e9d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:46:16.286253 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.286231    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-home" (OuterVolumeSpecName: "home") pod "0959a421-9458-4703-a0eb-ac946ac64e9d" (UID: "0959a421-9458-4703-a0eb-ac946ac64e9d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:46:16.286407 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.286275    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-model-cache\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:46:16.286407 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.286295    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-tmp-dir\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:46:16.288044 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.288015    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0959a421-9458-4703-a0eb-ac946ac64e9d-kube-api-access-bv9gq" (OuterVolumeSpecName: "kube-api-access-bv9gq") pod "0959a421-9458-4703-a0eb-ac946ac64e9d" (UID: "0959a421-9458-4703-a0eb-ac946ac64e9d"). InnerVolumeSpecName "kube-api-access-bv9gq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:46:16.288171 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.288067    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-dshm" (OuterVolumeSpecName: "dshm") pod "0959a421-9458-4703-a0eb-ac946ac64e9d" (UID: "0959a421-9458-4703-a0eb-ac946ac64e9d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:46:16.288171 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.288075    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0959a421-9458-4703-a0eb-ac946ac64e9d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0959a421-9458-4703-a0eb-ac946ac64e9d" (UID: "0959a421-9458-4703-a0eb-ac946ac64e9d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:46:16.386778 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.386751    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-home\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:46:16.386778 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.386772    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0959a421-9458-4703-a0eb-ac946ac64e9d-tls-certs\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:46:16.386778 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.386782    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0959a421-9458-4703-a0eb-ac946ac64e9d-dshm\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:46:16.386952 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.386790    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bv9gq\" (UniqueName: \"kubernetes.io/projected/0959a421-9458-4703-a0eb-ac946ac64e9d-kube-api-access-bv9gq\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:46:16.926200 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.926165    2556 generic.go:356] "Generic (PLEG): container finished" podID="0959a421-9458-4703-a0eb-ac946ac64e9d" containerID="07425bcb4af17dae8371eabd7fa079631afd747328fb110a4229ba5ddb15c1a9" exitCode=0
Aug 05 17:46:16.926200 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.926185    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8" event={"ID":"0959a421-9458-4703-a0eb-ac946ac64e9d","Type":"ContainerDied","Data":"07425bcb4af17dae8371eabd7fa079631afd747328fb110a4229ba5ddb15c1a9"}
Aug 05 17:46:16.926660 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.926206    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"
Aug 05 17:46:16.926660 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.926226    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8" event={"ID":"0959a421-9458-4703-a0eb-ac946ac64e9d","Type":"ContainerDied","Data":"45ddd9d5a2bf7177e216a87895b91addd1a5b8678edf60823e017a6af25b07ae"}
Aug 05 17:46:16.926660 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.926243    2556 scope.go:117] "RemoveContainer" containerID="07425bcb4af17dae8371eabd7fa079631afd747328fb110a4229ba5ddb15c1a9"
Aug 05 17:46:16.936263 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.936247    2556 scope.go:117] "RemoveContainer" containerID="07425bcb4af17dae8371eabd7fa079631afd747328fb110a4229ba5ddb15c1a9"
Aug 05 17:46:16.936494 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:46:16.936476    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07425bcb4af17dae8371eabd7fa079631afd747328fb110a4229ba5ddb15c1a9\": container with ID starting with 07425bcb4af17dae8371eabd7fa079631afd747328fb110a4229ba5ddb15c1a9 not found: ID does not exist" containerID="07425bcb4af17dae8371eabd7fa079631afd747328fb110a4229ba5ddb15c1a9"
Aug 05 17:46:16.936544 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.936503    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07425bcb4af17dae8371eabd7fa079631afd747328fb110a4229ba5ddb15c1a9"} err="failed to get container status \"07425bcb4af17dae8371eabd7fa079631afd747328fb110a4229ba5ddb15c1a9\": rpc error: code = NotFound desc = could not find container \"07425bcb4af17dae8371eabd7fa079631afd747328fb110a4229ba5ddb15c1a9\": container with ID starting with 07425bcb4af17dae8371eabd7fa079631afd747328fb110a4229ba5ddb15c1a9 not found: ID does not exist"
Aug 05 17:46:16.948000 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.947977    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"]
Aug 05 17:46:16.950661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:16.950639    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-0db46b7f-kserve-6dc4d594bc-cm7q8"]
Aug 05 17:46:17.096804 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.096772    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0959a421-9458-4703-a0eb-ac946ac64e9d" path="/var/lib/kubelet/pods/0959a421-9458-4703-a0eb-ac946ac64e9d/volumes"
Aug 05 17:46:17.431057 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.430984    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m" podUID="0a65a694-e8e5-4880-8539-f3d71ce31810" containerName="main" containerID="cri-o://6998ed96aa4e18de5ebd66e6f49595af1df605568f59949d27d21b84eb804be1" gracePeriod=45
Aug 05 17:46:17.678239 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.678215    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:46:17.796396 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.796362    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-model-cache\") pod \"0a65a694-e8e5-4880-8539-f3d71ce31810\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") "
Aug 05 17:46:17.796556 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.796425    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2g2hl\" (UniqueName: \"kubernetes.io/projected/0a65a694-e8e5-4880-8539-f3d71ce31810-kube-api-access-2g2hl\") pod \"0a65a694-e8e5-4880-8539-f3d71ce31810\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") "
Aug 05 17:46:17.796556 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.796452    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-dshm\") pod \"0a65a694-e8e5-4880-8539-f3d71ce31810\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") "
Aug 05 17:46:17.796556 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.796481    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-home\") pod \"0a65a694-e8e5-4880-8539-f3d71ce31810\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") "
Aug 05 17:46:17.796556 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.796540    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-tmp-dir\") pod \"0a65a694-e8e5-4880-8539-f3d71ce31810\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") "
Aug 05 17:46:17.796754 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.796575    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a65a694-e8e5-4880-8539-f3d71ce31810-tls-certs\") pod \"0a65a694-e8e5-4880-8539-f3d71ce31810\" (UID: \"0a65a694-e8e5-4880-8539-f3d71ce31810\") "
Aug 05 17:46:17.796754 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.796655    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-model-cache" (OuterVolumeSpecName: "model-cache") pod "0a65a694-e8e5-4880-8539-f3d71ce31810" (UID: "0a65a694-e8e5-4880-8539-f3d71ce31810"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:46:17.796858 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.796760    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-home" (OuterVolumeSpecName: "home") pod "0a65a694-e8e5-4880-8539-f3d71ce31810" (UID: "0a65a694-e8e5-4880-8539-f3d71ce31810"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:46:17.796858 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.796833    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0a65a694-e8e5-4880-8539-f3d71ce31810" (UID: "0a65a694-e8e5-4880-8539-f3d71ce31810"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:46:17.796941 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.796884    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-model-cache\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:46:17.796941 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.796900    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-home\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:46:17.798447 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.798423    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-dshm" (OuterVolumeSpecName: "dshm") pod "0a65a694-e8e5-4880-8539-f3d71ce31810" (UID: "0a65a694-e8e5-4880-8539-f3d71ce31810"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:46:17.798570 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.798488    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a65a694-e8e5-4880-8539-f3d71ce31810-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0a65a694-e8e5-4880-8539-f3d71ce31810" (UID: "0a65a694-e8e5-4880-8539-f3d71ce31810"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:46:17.798644 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.798572    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a65a694-e8e5-4880-8539-f3d71ce31810-kube-api-access-2g2hl" (OuterVolumeSpecName: "kube-api-access-2g2hl") pod "0a65a694-e8e5-4880-8539-f3d71ce31810" (UID: "0a65a694-e8e5-4880-8539-f3d71ce31810"). InnerVolumeSpecName "kube-api-access-2g2hl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:46:17.898061 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.898034    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2g2hl\" (UniqueName: \"kubernetes.io/projected/0a65a694-e8e5-4880-8539-f3d71ce31810-kube-api-access-2g2hl\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:46:17.898061 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.898057    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-dshm\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:46:17.898237 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.898089    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a65a694-e8e5-4880-8539-f3d71ce31810-tmp-dir\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:46:17.898237 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.898101    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a65a694-e8e5-4880-8539-f3d71ce31810-tls-certs\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:46:17.932769 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.932737    2556 generic.go:356] "Generic (PLEG): container finished" podID="0a65a694-e8e5-4880-8539-f3d71ce31810" containerID="6998ed96aa4e18de5ebd66e6f49595af1df605568f59949d27d21b84eb804be1" exitCode=0
Aug 05 17:46:17.933065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.932785    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"
Aug 05 17:46:17.933065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.932806    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m" event={"ID":"0a65a694-e8e5-4880-8539-f3d71ce31810","Type":"ContainerDied","Data":"6998ed96aa4e18de5ebd66e6f49595af1df605568f59949d27d21b84eb804be1"}
Aug 05 17:46:17.933065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.932842    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m" event={"ID":"0a65a694-e8e5-4880-8539-f3d71ce31810","Type":"ContainerDied","Data":"a41b59cf92de7d8fb7a6fff7bd0c90e12118fd4d267d5499ddf823eaf76e3329"}
Aug 05 17:46:17.933065 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.932864    2556 scope.go:117] "RemoveContainer" containerID="6998ed96aa4e18de5ebd66e6f49595af1df605568f59949d27d21b84eb804be1"
Aug 05 17:46:17.942181 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.942166    2556 scope.go:117] "RemoveContainer" containerID="6998ed96aa4e18de5ebd66e6f49595af1df605568f59949d27d21b84eb804be1"
Aug 05 17:46:17.942422 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:46:17.942405    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6998ed96aa4e18de5ebd66e6f49595af1df605568f59949d27d21b84eb804be1\": container with ID starting with 6998ed96aa4e18de5ebd66e6f49595af1df605568f59949d27d21b84eb804be1 not found: ID does not exist" containerID="6998ed96aa4e18de5ebd66e6f49595af1df605568f59949d27d21b84eb804be1"
Aug 05 17:46:17.942496 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.942432    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6998ed96aa4e18de5ebd66e6f49595af1df605568f59949d27d21b84eb804be1"} err="failed to get container status \"6998ed96aa4e18de5ebd66e6f49595af1df605568f59949d27d21b84eb804be1\": rpc error: code = NotFound desc = could not find container \"6998ed96aa4e18de5ebd66e6f49595af1df605568f59949d27d21b84eb804be1\": container with ID starting with 6998ed96aa4e18de5ebd66e6f49595af1df605568f59949d27d21b84eb804be1 not found: ID does not exist"
Aug 05 17:46:17.953177 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.953141    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"]
Aug 05 17:46:17.956392 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:17.956372    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-0db46b7f-kserve-5cb6549fb7-qrk2m"]
Aug 05 17:46:19.096985 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:46:19.096952    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a65a694-e8e5-4880-8539-f3d71ce31810" path="/var/lib/kubelet/pods/0a65a694-e8e5-4880-8539-f3d71ce31810/volumes"
Aug 05 17:50:02.665468 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.665433    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"]
Aug 05 17:50:02.666009 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.665993    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0959a421-9458-4703-a0eb-ac946ac64e9d" containerName="main"
Aug 05 17:50:02.666057 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.666012    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0959a421-9458-4703-a0eb-ac946ac64e9d" containerName="main"
Aug 05 17:50:02.666057 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.666023    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a65a694-e8e5-4880-8539-f3d71ce31810" containerName="main"
Aug 05 17:50:02.666057 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.666032    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a65a694-e8e5-4880-8539-f3d71ce31810" containerName="main"
Aug 05 17:50:02.666199 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.666118    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0959a421-9458-4703-a0eb-ac946ac64e9d" containerName="main"
Aug 05 17:50:02.666199 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.666133    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a65a694-e8e5-4880-8539-f3d71ce31810" containerName="main"
Aug 05 17:50:02.669459 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.669440    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:02.672200 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.672177    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ql6kj\""
Aug 05 17:50:02.672200 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.672193    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fbbe1196-serving-cert\""
Aug 05 17:50:02.678552 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.678529    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"]
Aug 05 17:50:02.702614 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.702587    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4drb\" (UniqueName: \"kubernetes.io/projected/b0877463-a087-45e5-a0d2-424ee34716f7-kube-api-access-b4drb\") pod \"maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s\" (UID: \"b0877463-a087-45e5-a0d2-424ee34716f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:02.702728 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.702621    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0877463-a087-45e5-a0d2-424ee34716f7-openshift-service-ca\") pod \"maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s\" (UID: \"b0877463-a087-45e5-a0d2-424ee34716f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:02.702728 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.702657    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0877463-a087-45e5-a0d2-424ee34716f7-maas-api-tls\") pod \"maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s\" (UID: \"b0877463-a087-45e5-a0d2-424ee34716f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:02.803865 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.803830    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0877463-a087-45e5-a0d2-424ee34716f7-maas-api-tls\") pod \"maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s\" (UID: \"b0877463-a087-45e5-a0d2-424ee34716f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:02.804032 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.803909    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4drb\" (UniqueName: \"kubernetes.io/projected/b0877463-a087-45e5-a0d2-424ee34716f7-kube-api-access-b4drb\") pod \"maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s\" (UID: \"b0877463-a087-45e5-a0d2-424ee34716f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:02.804032 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.803935    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0877463-a087-45e5-a0d2-424ee34716f7-openshift-service-ca\") pod \"maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s\" (UID: \"b0877463-a087-45e5-a0d2-424ee34716f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:02.804170 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:50:02.804028    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-serving-cert: secret "maas-api-e2e-ait-fbbe1196-serving-cert" not found
Aug 05 17:50:02.804170 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:50:02.804125    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0877463-a087-45e5-a0d2-424ee34716f7-maas-api-tls podName:b0877463-a087-45e5-a0d2-424ee34716f7 nodeName:}" failed. No retries permitted until 2026-08-05 17:50:03.304100936 +0000 UTC m=+1816.722415677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b0877463-a087-45e5-a0d2-424ee34716f7-maas-api-tls") pod "maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s" (UID: "b0877463-a087-45e5-a0d2-424ee34716f7") : secret "maas-api-e2e-ait-fbbe1196-serving-cert" not found
Aug 05 17:50:02.804683 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.804663    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0877463-a087-45e5-a0d2-424ee34716f7-openshift-service-ca\") pod \"maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s\" (UID: \"b0877463-a087-45e5-a0d2-424ee34716f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:02.818935 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:02.818911    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4drb\" (UniqueName: \"kubernetes.io/projected/b0877463-a087-45e5-a0d2-424ee34716f7-kube-api-access-b4drb\") pod \"maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s\" (UID: \"b0877463-a087-45e5-a0d2-424ee34716f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:03.306722 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.306690    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0877463-a087-45e5-a0d2-424ee34716f7-maas-api-tls\") pod \"maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s\" (UID: \"b0877463-a087-45e5-a0d2-424ee34716f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:03.309749 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.309720    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0877463-a087-45e5-a0d2-424ee34716f7-maas-api-tls\") pod \"maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s\" (UID: \"b0877463-a087-45e5-a0d2-424ee34716f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:03.351517 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.351492    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"]
Aug 05 17:50:03.355047 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.355030    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"
Aug 05 17:50:03.357367 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.357345    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-fbbe1196\""
Aug 05 17:50:03.357479 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.357370    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-fbbe1196-dockercfg-nds2m\""
Aug 05 17:50:03.365476 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.365449    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"]
Aug 05 17:50:03.390832 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.390810    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"]
Aug 05 17:50:03.394034 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.394018    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"
Aug 05 17:50:03.402911 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.402890    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"]
Aug 05 17:50:03.407304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.407281    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90714695-c33f-41d0-9c38-c36d40b07b79-plugins-config-volume\") pod \"payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq\" (UID: \"90714695-c33f-41d0-9c38-c36d40b07b79\") " pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"
Aug 05 17:50:03.407405 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.407342    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcxz2\" (UniqueName: \"kubernetes.io/projected/ce244052-b225-4029-b593-1923dd075cef-kube-api-access-rcxz2\") pod \"payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6\" (UID: \"ce244052-b225-4029-b593-1923dd075cef\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"
Aug 05 17:50:03.407473 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.407445    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce244052-b225-4029-b593-1923dd075cef-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6\" (UID: \"ce244052-b225-4029-b593-1923dd075cef\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"
Aug 05 17:50:03.407535 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.407472    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cdr6\" (UniqueName: \"kubernetes.io/projected/90714695-c33f-41d0-9c38-c36d40b07b79-kube-api-access-2cdr6\") pod \"payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq\" (UID: \"90714695-c33f-41d0-9c38-c36d40b07b79\") " pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"
Aug 05 17:50:03.508196 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.508131    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90714695-c33f-41d0-9c38-c36d40b07b79-plugins-config-volume\") pod \"payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq\" (UID: \"90714695-c33f-41d0-9c38-c36d40b07b79\") " pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"
Aug 05 17:50:03.508365 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.508241    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcxz2\" (UniqueName: \"kubernetes.io/projected/ce244052-b225-4029-b593-1923dd075cef-kube-api-access-rcxz2\") pod \"payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6\" (UID: \"ce244052-b225-4029-b593-1923dd075cef\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"
Aug 05 17:50:03.508365 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.508345    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce244052-b225-4029-b593-1923dd075cef-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6\" (UID: \"ce244052-b225-4029-b593-1923dd075cef\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"
Aug 05 17:50:03.508521 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.508374    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cdr6\" (UniqueName: \"kubernetes.io/projected/90714695-c33f-41d0-9c38-c36d40b07b79-kube-api-access-2cdr6\") pod \"payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq\" (UID: \"90714695-c33f-41d0-9c38-c36d40b07b79\") " pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"
Aug 05 17:50:03.509035 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.509011    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90714695-c33f-41d0-9c38-c36d40b07b79-plugins-config-volume\") pod \"payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq\" (UID: \"90714695-c33f-41d0-9c38-c36d40b07b79\") " pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"
Aug 05 17:50:03.509131 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.509114    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce244052-b225-4029-b593-1923dd075cef-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6\" (UID: \"ce244052-b225-4029-b593-1923dd075cef\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"
Aug 05 17:50:03.516500 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.516471    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cdr6\" (UniqueName: \"kubernetes.io/projected/90714695-c33f-41d0-9c38-c36d40b07b79-kube-api-access-2cdr6\") pod \"payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq\" (UID: \"90714695-c33f-41d0-9c38-c36d40b07b79\") " pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"
Aug 05 17:50:03.516622 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.516591    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcxz2\" (UniqueName: \"kubernetes.io/projected/ce244052-b225-4029-b593-1923dd075cef-kube-api-access-rcxz2\") pod \"payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6\" (UID: \"ce244052-b225-4029-b593-1923dd075cef\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"
Aug 05 17:50:03.582568 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.582497    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:03.666535 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.666489    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"
Aug 05 17:50:03.704768 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.704736    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"
Aug 05 17:50:03.723540 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.722687    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"]
Aug 05 17:50:03.727350 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.727318    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:50:03.815566 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.815393    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"]
Aug 05 17:50:03.818107 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:50:03.818082    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90714695_c33f_41d0_9c38_c36d40b07b79.slice/crio-2ff67d861cbc877020a8953f68173cbdd934ca8db50718a446654b98f9304127 WatchSource:0}: Error finding container 2ff67d861cbc877020a8953f68173cbdd934ca8db50718a446654b98f9304127: Status 404 returned error can't find the container with id 2ff67d861cbc877020a8953f68173cbdd934ca8db50718a446654b98f9304127
Aug 05 17:50:03.859665 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.859641    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"]
Aug 05 17:50:03.872363 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:50:03.872329    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce244052_b225_4029_b593_1923dd075cef.slice/crio-32a7743b54e0c2352d52f8a558bab191b62014ca95275a4609966b27f983af1a WatchSource:0}: Error finding container 32a7743b54e0c2352d52f8a558bab191b62014ca95275a4609966b27f983af1a: Status 404 returned error can't find the container with id 32a7743b54e0c2352d52f8a558bab191b62014ca95275a4609966b27f983af1a
Aug 05 17:50:03.958214 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.958181    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq" event={"ID":"90714695-c33f-41d0-9c38-c36d40b07b79","Type":"ContainerStarted","Data":"1783e61c025c47734debb028d0fc6305fed3c459fec10ecfcfbc431cb5af1108"}
Aug 05 17:50:03.958356 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.958231    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq" event={"ID":"90714695-c33f-41d0-9c38-c36d40b07b79","Type":"ContainerStarted","Data":"2ff67d861cbc877020a8953f68173cbdd934ca8db50718a446654b98f9304127"}
Aug 05 17:50:03.958356 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.958266    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"
Aug 05 17:50:03.959897 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.959873    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6" event={"ID":"ce244052-b225-4029-b593-1923dd075cef","Type":"ContainerStarted","Data":"d7a9b26cc7de69d9bcb805fd31aaa315149893ae0df52b5ee5d7c9e5665f7a3d"}
Aug 05 17:50:03.960016 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.959915    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6" event={"ID":"ce244052-b225-4029-b593-1923dd075cef","Type":"ContainerStarted","Data":"32a7743b54e0c2352d52f8a558bab191b62014ca95275a4609966b27f983af1a"}
Aug 05 17:50:03.960016 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.959944    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"
Aug 05 17:50:03.961251 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.961224    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s" event={"ID":"b0877463-a087-45e5-a0d2-424ee34716f7","Type":"ContainerStarted","Data":"71f3fe45fea24f396331613bfa32c265ac108a57630b292469abe732b1a342f7"}
Aug 05 17:50:03.974457 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.974412    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq" podStartSLOduration=0.974397197 podStartE2EDuration="974.397197ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:50:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:50:03.972548761 +0000 UTC m=+1817.390863556" watchObservedRunningTime="2026-08-05 17:50:03.974397197 +0000 UTC m=+1817.392711932"
Aug 05 17:50:03.987895 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:03.987856    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6" podStartSLOduration=0.987840466 podStartE2EDuration="987.840466ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:50:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:50:03.986073513 +0000 UTC m=+1817.404388287" watchObservedRunningTime="2026-08-05 17:50:03.987840466 +0000 UTC m=+1817.406155223"
Aug 05 17:50:04.968557 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:04.968518    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s" event={"ID":"b0877463-a087-45e5-a0d2-424ee34716f7","Type":"ContainerStarted","Data":"ec14fa69c19ef7eb6a3311d8fcb36e2f7cc63d11ea4a3ab91bcd7905b9942e7f"}
Aug 05 17:50:04.968989 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:04.968896    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:04.985431 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:04.985389    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s" podStartSLOduration=1.962045813 podStartE2EDuration="2.98537466s" totalImagesPullingTime="1.023328847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:50:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:50:04.983550328 +0000 UTC m=+1818.401865098" watchObservedRunningTime="2026-08-05 17:50:04.98537466 +0000 UTC m=+1818.403689417"
Aug 05 17:50:10.978208 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:10.978178    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:14.970352 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:14.970310    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"
Aug 05 17:50:14.970837 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:14.970394    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"
Aug 05 17:50:43.332898 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.332861    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"]
Aug 05 17:50:43.333504 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.333058    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s" podUID="b0877463-a087-45e5-a0d2-424ee34716f7" containerName="maas-api" containerID="cri-o://ec14fa69c19ef7eb6a3311d8fcb36e2f7cc63d11ea4a3ab91bcd7905b9942e7f" gracePeriod=30
Aug 05 17:50:43.393776 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.393747    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"]
Aug 05 17:50:43.394005 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.393980    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq" podUID="90714695-c33f-41d0-9c38-c36d40b07b79" containerName="payload-processing" containerID="cri-o://1783e61c025c47734debb028d0fc6305fed3c459fec10ecfcfbc431cb5af1108" gracePeriod=30
Aug 05 17:50:43.401505 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.401479    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"]
Aug 05 17:50:43.402261 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.401706    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6" podUID="ce244052-b225-4029-b593-1923dd075cef" containerName="payload-pre-processing" containerID="cri-o://d7a9b26cc7de69d9bcb805fd31aaa315149893ae0df52b5ee5d7c9e5665f7a3d" gracePeriod=30
Aug 05 17:50:43.659326 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.659290    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"
Aug 05 17:50:43.690279 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.690251    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:43.710266 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.710031    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"
Aug 05 17:50:43.732034 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.732005    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cdr6\" (UniqueName: \"kubernetes.io/projected/90714695-c33f-41d0-9c38-c36d40b07b79-kube-api-access-2cdr6\") pod \"90714695-c33f-41d0-9c38-c36d40b07b79\" (UID: \"90714695-c33f-41d0-9c38-c36d40b07b79\") "
Aug 05 17:50:43.732186 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.732168    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90714695-c33f-41d0-9c38-c36d40b07b79-plugins-config-volume\") pod \"90714695-c33f-41d0-9c38-c36d40b07b79\" (UID: \"90714695-c33f-41d0-9c38-c36d40b07b79\") "
Aug 05 17:50:43.732479 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.732456    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90714695-c33f-41d0-9c38-c36d40b07b79-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "90714695-c33f-41d0-9c38-c36d40b07b79" (UID: "90714695-c33f-41d0-9c38-c36d40b07b79"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:50:43.734040 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.734013    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90714695-c33f-41d0-9c38-c36d40b07b79-kube-api-access-2cdr6" (OuterVolumeSpecName: "kube-api-access-2cdr6") pod "90714695-c33f-41d0-9c38-c36d40b07b79" (UID: "90714695-c33f-41d0-9c38-c36d40b07b79"). InnerVolumeSpecName "kube-api-access-2cdr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:50:43.833518 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.833488    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0877463-a087-45e5-a0d2-424ee34716f7-openshift-service-ca\") pod \"b0877463-a087-45e5-a0d2-424ee34716f7\" (UID: \"b0877463-a087-45e5-a0d2-424ee34716f7\") "
Aug 05 17:50:43.833658 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.833548    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4drb\" (UniqueName: \"kubernetes.io/projected/b0877463-a087-45e5-a0d2-424ee34716f7-kube-api-access-b4drb\") pod \"b0877463-a087-45e5-a0d2-424ee34716f7\" (UID: \"b0877463-a087-45e5-a0d2-424ee34716f7\") "
Aug 05 17:50:43.833658 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.833596    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce244052-b225-4029-b593-1923dd075cef-plugins-config-volume\") pod \"ce244052-b225-4029-b593-1923dd075cef\" (UID: \"ce244052-b225-4029-b593-1923dd075cef\") "
Aug 05 17:50:43.833658 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.833625    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0877463-a087-45e5-a0d2-424ee34716f7-maas-api-tls\") pod \"b0877463-a087-45e5-a0d2-424ee34716f7\" (UID: \"b0877463-a087-45e5-a0d2-424ee34716f7\") "
Aug 05 17:50:43.833658 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.833650    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcxz2\" (UniqueName: \"kubernetes.io/projected/ce244052-b225-4029-b593-1923dd075cef-kube-api-access-rcxz2\") pod \"ce244052-b225-4029-b593-1923dd075cef\" (UID: \"ce244052-b225-4029-b593-1923dd075cef\") "
Aug 05 17:50:43.833888 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.833852    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0877463-a087-45e5-a0d2-424ee34716f7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b0877463-a087-45e5-a0d2-424ee34716f7" (UID: "b0877463-a087-45e5-a0d2-424ee34716f7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:50:43.833949 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.833918    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90714695-c33f-41d0-9c38-c36d40b07b79-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:50:43.833949 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.833937    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cdr6\" (UniqueName: \"kubernetes.io/projected/90714695-c33f-41d0-9c38-c36d40b07b79-kube-api-access-2cdr6\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:50:43.834033 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.833944    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce244052-b225-4029-b593-1923dd075cef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ce244052-b225-4029-b593-1923dd075cef" (UID: "ce244052-b225-4029-b593-1923dd075cef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:50:43.835748 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.835719    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce244052-b225-4029-b593-1923dd075cef-kube-api-access-rcxz2" (OuterVolumeSpecName: "kube-api-access-rcxz2") pod "ce244052-b225-4029-b593-1923dd075cef" (UID: "ce244052-b225-4029-b593-1923dd075cef"). InnerVolumeSpecName "kube-api-access-rcxz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:50:43.835840 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.835818    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0877463-a087-45e5-a0d2-424ee34716f7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b0877463-a087-45e5-a0d2-424ee34716f7" (UID: "b0877463-a087-45e5-a0d2-424ee34716f7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:50:43.835882 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.835863    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0877463-a087-45e5-a0d2-424ee34716f7-kube-api-access-b4drb" (OuterVolumeSpecName: "kube-api-access-b4drb") pod "b0877463-a087-45e5-a0d2-424ee34716f7" (UID: "b0877463-a087-45e5-a0d2-424ee34716f7"). InnerVolumeSpecName "kube-api-access-b4drb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:50:43.935008 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.934950    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0877463-a087-45e5-a0d2-424ee34716f7-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:50:43.935008 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.934972    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4drb\" (UniqueName: \"kubernetes.io/projected/b0877463-a087-45e5-a0d2-424ee34716f7-kube-api-access-b4drb\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:50:43.935008 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.934982    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce244052-b225-4029-b593-1923dd075cef-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:50:43.935008 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.934990    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0877463-a087-45e5-a0d2-424ee34716f7-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:50:43.935008 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:43.935001    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcxz2\" (UniqueName: \"kubernetes.io/projected/ce244052-b225-4029-b593-1923dd075cef-kube-api-access-rcxz2\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:50:44.147247 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.147209    2556 generic.go:356] "Generic (PLEG): container finished" podID="90714695-c33f-41d0-9c38-c36d40b07b79" containerID="1783e61c025c47734debb028d0fc6305fed3c459fec10ecfcfbc431cb5af1108" exitCode=0
Aug 05 17:50:44.147408 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.147262    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"
Aug 05 17:50:44.147408 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.147305    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq" event={"ID":"90714695-c33f-41d0-9c38-c36d40b07b79","Type":"ContainerDied","Data":"1783e61c025c47734debb028d0fc6305fed3c459fec10ecfcfbc431cb5af1108"}
Aug 05 17:50:44.147408 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.147367    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq" event={"ID":"90714695-c33f-41d0-9c38-c36d40b07b79","Type":"ContainerDied","Data":"2ff67d861cbc877020a8953f68173cbdd934ca8db50718a446654b98f9304127"}
Aug 05 17:50:44.147408 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.147400    2556 scope.go:117] "RemoveContainer" containerID="1783e61c025c47734debb028d0fc6305fed3c459fec10ecfcfbc431cb5af1108"
Aug 05 17:50:44.148969 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.148942    2556 generic.go:356] "Generic (PLEG): container finished" podID="ce244052-b225-4029-b593-1923dd075cef" containerID="d7a9b26cc7de69d9bcb805fd31aaa315149893ae0df52b5ee5d7c9e5665f7a3d" exitCode=0
Aug 05 17:50:44.149076 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.148982    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"
Aug 05 17:50:44.149076 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.149026    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6" event={"ID":"ce244052-b225-4029-b593-1923dd075cef","Type":"ContainerDied","Data":"d7a9b26cc7de69d9bcb805fd31aaa315149893ae0df52b5ee5d7c9e5665f7a3d"}
Aug 05 17:50:44.149225 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.149076    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6" event={"ID":"ce244052-b225-4029-b593-1923dd075cef","Type":"ContainerDied","Data":"32a7743b54e0c2352d52f8a558bab191b62014ca95275a4609966b27f983af1a"}
Aug 05 17:50:44.150433 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.150408    2556 generic.go:356] "Generic (PLEG): container finished" podID="b0877463-a087-45e5-a0d2-424ee34716f7" containerID="ec14fa69c19ef7eb6a3311d8fcb36e2f7cc63d11ea4a3ab91bcd7905b9942e7f" exitCode=0
Aug 05 17:50:44.150532 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.150450    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s" event={"ID":"b0877463-a087-45e5-a0d2-424ee34716f7","Type":"ContainerDied","Data":"ec14fa69c19ef7eb6a3311d8fcb36e2f7cc63d11ea4a3ab91bcd7905b9942e7f"}
Aug 05 17:50:44.150532 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.150460    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"
Aug 05 17:50:44.150532 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.150490    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s" event={"ID":"b0877463-a087-45e5-a0d2-424ee34716f7","Type":"ContainerDied","Data":"71f3fe45fea24f396331613bfa32c265ac108a57630b292469abe732b1a342f7"}
Aug 05 17:50:44.157966 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.157948    2556 scope.go:117] "RemoveContainer" containerID="1783e61c025c47734debb028d0fc6305fed3c459fec10ecfcfbc431cb5af1108"
Aug 05 17:50:44.158264 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:50:44.158244    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1783e61c025c47734debb028d0fc6305fed3c459fec10ecfcfbc431cb5af1108\": container with ID starting with 1783e61c025c47734debb028d0fc6305fed3c459fec10ecfcfbc431cb5af1108 not found: ID does not exist" containerID="1783e61c025c47734debb028d0fc6305fed3c459fec10ecfcfbc431cb5af1108"
Aug 05 17:50:44.158343 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.158272    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1783e61c025c47734debb028d0fc6305fed3c459fec10ecfcfbc431cb5af1108"} err="failed to get container status \"1783e61c025c47734debb028d0fc6305fed3c459fec10ecfcfbc431cb5af1108\": rpc error: code = NotFound desc = could not find container \"1783e61c025c47734debb028d0fc6305fed3c459fec10ecfcfbc431cb5af1108\": container with ID starting with 1783e61c025c47734debb028d0fc6305fed3c459fec10ecfcfbc431cb5af1108 not found: ID does not exist"
Aug 05 17:50:44.158343 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.158295    2556 scope.go:117] "RemoveContainer" containerID="d7a9b26cc7de69d9bcb805fd31aaa315149893ae0df52b5ee5d7c9e5665f7a3d"
Aug 05 17:50:44.168800 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.168785    2556 scope.go:117] "RemoveContainer" containerID="d7a9b26cc7de69d9bcb805fd31aaa315149893ae0df52b5ee5d7c9e5665f7a3d"
Aug 05 17:50:44.169108 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:50:44.169086    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7a9b26cc7de69d9bcb805fd31aaa315149893ae0df52b5ee5d7c9e5665f7a3d\": container with ID starting with d7a9b26cc7de69d9bcb805fd31aaa315149893ae0df52b5ee5d7c9e5665f7a3d not found: ID does not exist" containerID="d7a9b26cc7de69d9bcb805fd31aaa315149893ae0df52b5ee5d7c9e5665f7a3d"
Aug 05 17:50:44.169177 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.169122    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a9b26cc7de69d9bcb805fd31aaa315149893ae0df52b5ee5d7c9e5665f7a3d"} err="failed to get container status \"d7a9b26cc7de69d9bcb805fd31aaa315149893ae0df52b5ee5d7c9e5665f7a3d\": rpc error: code = NotFound desc = could not find container \"d7a9b26cc7de69d9bcb805fd31aaa315149893ae0df52b5ee5d7c9e5665f7a3d\": container with ID starting with d7a9b26cc7de69d9bcb805fd31aaa315149893ae0df52b5ee5d7c9e5665f7a3d not found: ID does not exist"
Aug 05 17:50:44.169177 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.169144    2556 scope.go:117] "RemoveContainer" containerID="ec14fa69c19ef7eb6a3311d8fcb36e2f7cc63d11ea4a3ab91bcd7905b9942e7f"
Aug 05 17:50:44.177698 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.177682    2556 scope.go:117] "RemoveContainer" containerID="ec14fa69c19ef7eb6a3311d8fcb36e2f7cc63d11ea4a3ab91bcd7905b9942e7f"
Aug 05 17:50:44.177926 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:50:44.177909    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec14fa69c19ef7eb6a3311d8fcb36e2f7cc63d11ea4a3ab91bcd7905b9942e7f\": container with ID starting with ec14fa69c19ef7eb6a3311d8fcb36e2f7cc63d11ea4a3ab91bcd7905b9942e7f not found: ID does not exist" containerID="ec14fa69c19ef7eb6a3311d8fcb36e2f7cc63d11ea4a3ab91bcd7905b9942e7f"
Aug 05 17:50:44.177989 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.177932    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec14fa69c19ef7eb6a3311d8fcb36e2f7cc63d11ea4a3ab91bcd7905b9942e7f"} err="failed to get container status \"ec14fa69c19ef7eb6a3311d8fcb36e2f7cc63d11ea4a3ab91bcd7905b9942e7f\": rpc error: code = NotFound desc = could not find container \"ec14fa69c19ef7eb6a3311d8fcb36e2f7cc63d11ea4a3ab91bcd7905b9942e7f\": container with ID starting with ec14fa69c19ef7eb6a3311d8fcb36e2f7cc63d11ea4a3ab91bcd7905b9942e7f not found: ID does not exist"
Aug 05 17:50:44.186011 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.185966    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"]
Aug 05 17:50:44.188850 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.188831    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fbbe1196-7fbd44bdcb-pbdw6"]
Aug 05 17:50:44.209897 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.209874    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"]
Aug 05 17:50:44.213407 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.213385    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fbbe1196-77f8db7f76-6n95s"]
Aug 05 17:50:44.233060 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.233038    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"]
Aug 05 17:50:44.236936 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:44.236916    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fbbe1196-b8c44b984-5fphq"]
Aug 05 17:50:45.099580 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:45.099545    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90714695-c33f-41d0-9c38-c36d40b07b79" path="/var/lib/kubelet/pods/90714695-c33f-41d0-9c38-c36d40b07b79/volumes"
Aug 05 17:50:45.100223 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:45.100204    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0877463-a087-45e5-a0d2-424ee34716f7" path="/var/lib/kubelet/pods/b0877463-a087-45e5-a0d2-424ee34716f7/volumes"
Aug 05 17:50:45.100643 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:45.100628    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce244052-b225-4029-b593-1923dd075cef" path="/var/lib/kubelet/pods/ce244052-b225-4029-b593-1923dd075cef/volumes"
Aug 05 17:50:47.664814 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.664775    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"]
Aug 05 17:50:47.665377 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.665356    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce244052-b225-4029-b593-1923dd075cef" containerName="payload-pre-processing"
Aug 05 17:50:47.665444 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.665380    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce244052-b225-4029-b593-1923dd075cef" containerName="payload-pre-processing"
Aug 05 17:50:47.665444 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.665420    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90714695-c33f-41d0-9c38-c36d40b07b79" containerName="payload-processing"
Aug 05 17:50:47.665444 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.665429    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="90714695-c33f-41d0-9c38-c36d40b07b79" containerName="payload-processing"
Aug 05 17:50:47.665444 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.665442    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0877463-a087-45e5-a0d2-424ee34716f7" containerName="maas-api"
Aug 05 17:50:47.665636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.665452    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0877463-a087-45e5-a0d2-424ee34716f7" containerName="maas-api"
Aug 05 17:50:47.665636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.665545    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce244052-b225-4029-b593-1923dd075cef" containerName="payload-pre-processing"
Aug 05 17:50:47.665636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.665557    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0877463-a087-45e5-a0d2-424ee34716f7" containerName="maas-api"
Aug 05 17:50:47.665636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.665565    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="90714695-c33f-41d0-9c38-c36d40b07b79" containerName="payload-processing"
Aug 05 17:50:47.672531 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.672503    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:50:47.677451 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.677432    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-41ad7fbb-serving-cert\""
Aug 05 17:50:47.679142 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.679120    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ql6kj\""
Aug 05 17:50:47.679333 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.679317    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"]
Aug 05 17:50:47.768174 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.768122    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d99800df-6988-4416-b4a7-249e156f922a-maas-api-tls\") pod \"maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb\" (UID: \"d99800df-6988-4416-b4a7-249e156f922a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:50:47.768174 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.768176    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7pxc\" (UniqueName: \"kubernetes.io/projected/d99800df-6988-4416-b4a7-249e156f922a-kube-api-access-j7pxc\") pod \"maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb\" (UID: \"d99800df-6988-4416-b4a7-249e156f922a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:50:47.768408 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.768290    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d99800df-6988-4416-b4a7-249e156f922a-openshift-service-ca\") pod \"maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb\" (UID: \"d99800df-6988-4416-b4a7-249e156f922a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:50:47.869037 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.868995    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d99800df-6988-4416-b4a7-249e156f922a-openshift-service-ca\") pod \"maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb\" (UID: \"d99800df-6988-4416-b4a7-249e156f922a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:50:47.869222 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.869121    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d99800df-6988-4416-b4a7-249e156f922a-maas-api-tls\") pod \"maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb\" (UID: \"d99800df-6988-4416-b4a7-249e156f922a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:50:47.869222 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.869164    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7pxc\" (UniqueName: \"kubernetes.io/projected/d99800df-6988-4416-b4a7-249e156f922a-kube-api-access-j7pxc\") pod \"maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb\" (UID: \"d99800df-6988-4416-b4a7-249e156f922a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:50:47.869347 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:50:47.869279    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-serving-cert: secret "maas-api-e2e-migrate-41ad7fbb-serving-cert" not found
Aug 05 17:50:47.869398 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:50:47.869366    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d99800df-6988-4416-b4a7-249e156f922a-maas-api-tls podName:d99800df-6988-4416-b4a7-249e156f922a nodeName:}" failed. No retries permitted until 2026-08-05 17:50:48.369343787 +0000 UTC m=+1861.787658526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d99800df-6988-4416-b4a7-249e156f922a-maas-api-tls") pod "maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb" (UID: "d99800df-6988-4416-b4a7-249e156f922a") : secret "maas-api-e2e-migrate-41ad7fbb-serving-cert" not found
Aug 05 17:50:47.869798 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.869776    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d99800df-6988-4416-b4a7-249e156f922a-openshift-service-ca\") pod \"maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb\" (UID: \"d99800df-6988-4416-b4a7-249e156f922a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:50:47.896099 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:47.896077    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7pxc\" (UniqueName: \"kubernetes.io/projected/d99800df-6988-4416-b4a7-249e156f922a-kube-api-access-j7pxc\") pod \"maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb\" (UID: \"d99800df-6988-4416-b4a7-249e156f922a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:50:48.375275 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.375238    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d99800df-6988-4416-b4a7-249e156f922a-maas-api-tls\") pod \"maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb\" (UID: \"d99800df-6988-4416-b4a7-249e156f922a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:50:48.377953 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.377920    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d99800df-6988-4416-b4a7-249e156f922a-maas-api-tls\") pod \"maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb\" (UID: \"d99800df-6988-4416-b4a7-249e156f922a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:50:48.480146 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.480116    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"]
Aug 05 17:50:48.483096 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.483079    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"
Aug 05 17:50:48.485548 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.485530    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-41ad7fbb-dockercfg-hbxcc\""
Aug 05 17:50:48.485615 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.485569    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-41ad7fbb\""
Aug 05 17:50:48.492409 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.492388    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"]
Aug 05 17:50:48.576630 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.576601    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8p5j\" (UniqueName: \"kubernetes.io/projected/e62ae463-4a0f-4da0-b10c-950ea6c542fb-kube-api-access-s8p5j\") pod \"payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp\" (UID: \"e62ae463-4a0f-4da0-b10c-950ea6c542fb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"
Aug 05 17:50:48.576836 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.576662    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e62ae463-4a0f-4da0-b10c-950ea6c542fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp\" (UID: \"e62ae463-4a0f-4da0-b10c-950ea6c542fb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"
Aug 05 17:50:48.586566 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.586543    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:50:48.677599 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.677569    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8p5j\" (UniqueName: \"kubernetes.io/projected/e62ae463-4a0f-4da0-b10c-950ea6c542fb-kube-api-access-s8p5j\") pod \"payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp\" (UID: \"e62ae463-4a0f-4da0-b10c-950ea6c542fb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"
Aug 05 17:50:48.677966 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.677672    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e62ae463-4a0f-4da0-b10c-950ea6c542fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp\" (UID: \"e62ae463-4a0f-4da0-b10c-950ea6c542fb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"
Aug 05 17:50:48.678650 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.678615    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e62ae463-4a0f-4da0-b10c-950ea6c542fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp\" (UID: \"e62ae463-4a0f-4da0-b10c-950ea6c542fb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"
Aug 05 17:50:48.687099 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.687070    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8p5j\" (UniqueName: \"kubernetes.io/projected/e62ae463-4a0f-4da0-b10c-950ea6c542fb-kube-api-access-s8p5j\") pod \"payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp\" (UID: \"e62ae463-4a0f-4da0-b10c-950ea6c542fb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"
Aug 05 17:50:48.727524 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.727486    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"]
Aug 05 17:50:48.729491 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:50:48.729470    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd99800df_6988_4416_b4a7_249e156f922a.slice/crio-4c125660b10206f9b9cba4ad187d4701b98c48bd5dcabce8554c5b87a704f684 WatchSource:0}: Error finding container 4c125660b10206f9b9cba4ad187d4701b98c48bd5dcabce8554c5b87a704f684: Status 404 returned error can't find the container with id 4c125660b10206f9b9cba4ad187d4701b98c48bd5dcabce8554c5b87a704f684
Aug 05 17:50:48.794376 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.794352    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"
Aug 05 17:50:48.932480 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:48.932455    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"]
Aug 05 17:50:48.934730 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:50:48.934701    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode62ae463_4a0f_4da0_b10c_950ea6c542fb.slice/crio-9e038e719ef28b511a54643aa719bdae49f12d0f513c17b3ef8ab05da04853c3 WatchSource:0}: Error finding container 9e038e719ef28b511a54643aa719bdae49f12d0f513c17b3ef8ab05da04853c3: Status 404 returned error can't find the container with id 9e038e719ef28b511a54643aa719bdae49f12d0f513c17b3ef8ab05da04853c3
Aug 05 17:50:49.175895 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:49.175864    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb" event={"ID":"d99800df-6988-4416-b4a7-249e156f922a","Type":"ContainerStarted","Data":"4c125660b10206f9b9cba4ad187d4701b98c48bd5dcabce8554c5b87a704f684"}
Aug 05 17:50:49.177350 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:49.177325    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp" event={"ID":"e62ae463-4a0f-4da0-b10c-950ea6c542fb","Type":"ContainerStarted","Data":"802a5139f9a050b524f6dc3d8fa46bf3816cd577ae10d3d093cb6ec68313c188"}
Aug 05 17:50:49.177475 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:49.177366    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp" event={"ID":"e62ae463-4a0f-4da0-b10c-950ea6c542fb","Type":"ContainerStarted","Data":"9e038e719ef28b511a54643aa719bdae49f12d0f513c17b3ef8ab05da04853c3"}
Aug 05 17:50:49.177475 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:49.177452    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"
Aug 05 17:50:49.192800 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:49.192592    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp" podStartSLOduration=1.192575345 podStartE2EDuration="1.192575345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:50:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:50:49.192143939 +0000 UTC m=+1862.610458696" watchObservedRunningTime="2026-08-05 17:50:49.192575345 +0000 UTC m=+1862.610890105"
Aug 05 17:50:50.183780 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:50.183744    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb" event={"ID":"d99800df-6988-4416-b4a7-249e156f922a","Type":"ContainerStarted","Data":"5a43eb57f95d4aa240fee9f1f965cbcdaec04b9f9f8807cce0221c6a5a5ff58d"}
Aug 05 17:50:50.201852 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:50.201807    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb" podStartSLOduration=2.707024336 podStartE2EDuration="3.201793258s" totalImagesPullingTime="494.768922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:50:50.200630133 +0000 UTC m=+1863.618944916" watchObservedRunningTime="2026-08-05 17:50:50.201793258 +0000 UTC m=+1863.620108055"
Aug 05 17:50:51.187691 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:51.187664    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:50:57.197364 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:50:57.197331    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:51:00.186052 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:00.186017    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"
Aug 05 17:51:19.042945 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.042912    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"]
Aug 05 17:51:19.043395 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.043126    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb" podUID="d99800df-6988-4416-b4a7-249e156f922a" containerName="maas-api" containerID="cri-o://5a43eb57f95d4aa240fee9f1f965cbcdaec04b9f9f8807cce0221c6a5a5ff58d" gracePeriod=30
Aug 05 17:51:19.121221 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.121112    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"]
Aug 05 17:51:19.133632 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.121312    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp" podUID="e62ae463-4a0f-4da0-b10c-950ea6c542fb" containerName="payload-pre-processing" containerID="cri-o://802a5139f9a050b524f6dc3d8fa46bf3816cd577ae10d3d093cb6ec68313c188" gracePeriod=30
Aug 05 17:51:19.321054 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.321015    2556 generic.go:356] "Generic (PLEG): container finished" podID="d99800df-6988-4416-b4a7-249e156f922a" containerID="5a43eb57f95d4aa240fee9f1f965cbcdaec04b9f9f8807cce0221c6a5a5ff58d" exitCode=0
Aug 05 17:51:19.321257 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.321078    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb" event={"ID":"d99800df-6988-4416-b4a7-249e156f922a","Type":"ContainerDied","Data":"5a43eb57f95d4aa240fee9f1f965cbcdaec04b9f9f8807cce0221c6a5a5ff58d"}
Aug 05 17:51:19.323106 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.323077    2556 generic.go:356] "Generic (PLEG): container finished" podID="e62ae463-4a0f-4da0-b10c-950ea6c542fb" containerID="802a5139f9a050b524f6dc3d8fa46bf3816cd577ae10d3d093cb6ec68313c188" exitCode=0
Aug 05 17:51:19.323267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.323182    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp" event={"ID":"e62ae463-4a0f-4da0-b10c-950ea6c542fb","Type":"ContainerDied","Data":"802a5139f9a050b524f6dc3d8fa46bf3816cd577ae10d3d093cb6ec68313c188"}
Aug 05 17:51:19.376350 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.376321    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:51:19.400687 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.400658    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"
Aug 05 17:51:19.412007 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.411975    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7pxc\" (UniqueName: \"kubernetes.io/projected/d99800df-6988-4416-b4a7-249e156f922a-kube-api-access-j7pxc\") pod \"d99800df-6988-4416-b4a7-249e156f922a\" (UID: \"d99800df-6988-4416-b4a7-249e156f922a\") "
Aug 05 17:51:19.412129 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.412039    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8p5j\" (UniqueName: \"kubernetes.io/projected/e62ae463-4a0f-4da0-b10c-950ea6c542fb-kube-api-access-s8p5j\") pod \"e62ae463-4a0f-4da0-b10c-950ea6c542fb\" (UID: \"e62ae463-4a0f-4da0-b10c-950ea6c542fb\") "
Aug 05 17:51:19.412129 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.412105    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e62ae463-4a0f-4da0-b10c-950ea6c542fb-plugins-config-volume\") pod \"e62ae463-4a0f-4da0-b10c-950ea6c542fb\" (UID: \"e62ae463-4a0f-4da0-b10c-950ea6c542fb\") "
Aug 05 17:51:19.412262 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.412186    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d99800df-6988-4416-b4a7-249e156f922a-openshift-service-ca\") pod \"d99800df-6988-4416-b4a7-249e156f922a\" (UID: \"d99800df-6988-4416-b4a7-249e156f922a\") "
Aug 05 17:51:19.412262 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.412239    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d99800df-6988-4416-b4a7-249e156f922a-maas-api-tls\") pod \"d99800df-6988-4416-b4a7-249e156f922a\" (UID: \"d99800df-6988-4416-b4a7-249e156f922a\") "
Aug 05 17:51:19.412608 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.412581    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d99800df-6988-4416-b4a7-249e156f922a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d99800df-6988-4416-b4a7-249e156f922a" (UID: "d99800df-6988-4416-b4a7-249e156f922a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:51:19.413020 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.412996    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e62ae463-4a0f-4da0-b10c-950ea6c542fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e62ae463-4a0f-4da0-b10c-950ea6c542fb" (UID: "e62ae463-4a0f-4da0-b10c-950ea6c542fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:51:19.414336 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.414309    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d99800df-6988-4416-b4a7-249e156f922a-kube-api-access-j7pxc" (OuterVolumeSpecName: "kube-api-access-j7pxc") pod "d99800df-6988-4416-b4a7-249e156f922a" (UID: "d99800df-6988-4416-b4a7-249e156f922a"). InnerVolumeSpecName "kube-api-access-j7pxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:51:19.414732 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.414704    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d99800df-6988-4416-b4a7-249e156f922a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d99800df-6988-4416-b4a7-249e156f922a" (UID: "d99800df-6988-4416-b4a7-249e156f922a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:51:19.414732 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.414715    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e62ae463-4a0f-4da0-b10c-950ea6c542fb-kube-api-access-s8p5j" (OuterVolumeSpecName: "kube-api-access-s8p5j") pod "e62ae463-4a0f-4da0-b10c-950ea6c542fb" (UID: "e62ae463-4a0f-4da0-b10c-950ea6c542fb"). InnerVolumeSpecName "kube-api-access-s8p5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:51:19.513440 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.513407    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7pxc\" (UniqueName: \"kubernetes.io/projected/d99800df-6988-4416-b4a7-249e156f922a-kube-api-access-j7pxc\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:51:19.513440 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.513443    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8p5j\" (UniqueName: \"kubernetes.io/projected/e62ae463-4a0f-4da0-b10c-950ea6c542fb-kube-api-access-s8p5j\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:51:19.513626 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.513456    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e62ae463-4a0f-4da0-b10c-950ea6c542fb-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:51:19.513626 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.513468    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d99800df-6988-4416-b4a7-249e156f922a-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:51:19.513626 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:19.513481    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d99800df-6988-4416-b4a7-249e156f922a-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:51:20.331349 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:20.331263    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb" event={"ID":"d99800df-6988-4416-b4a7-249e156f922a","Type":"ContainerDied","Data":"4c125660b10206f9b9cba4ad187d4701b98c48bd5dcabce8554c5b87a704f684"}
Aug 05 17:51:20.331349 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:20.331291    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"
Aug 05 17:51:20.331349 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:20.331333    2556 scope.go:117] "RemoveContainer" containerID="5a43eb57f95d4aa240fee9f1f965cbcdaec04b9f9f8807cce0221c6a5a5ff58d"
Aug 05 17:51:20.332908 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:20.332709    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp" event={"ID":"e62ae463-4a0f-4da0-b10c-950ea6c542fb","Type":"ContainerDied","Data":"9e038e719ef28b511a54643aa719bdae49f12d0f513c17b3ef8ab05da04853c3"}
Aug 05 17:51:20.332908 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:20.332787    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"
Aug 05 17:51:20.342544 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:20.342528    2556 scope.go:117] "RemoveContainer" containerID="802a5139f9a050b524f6dc3d8fa46bf3816cd577ae10d3d093cb6ec68313c188"
Aug 05 17:51:20.358337 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:20.358310    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"]
Aug 05 17:51:20.364779 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:20.364755    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-41ad7fbb-699f5c6468-l29jb"]
Aug 05 17:51:20.376812 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:20.376793    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"]
Aug 05 17:51:20.379626 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:20.379596    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-41ad7fbb-64dcbfd684-lt6sp"]
Aug 05 17:51:21.097108 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:21.097077    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d99800df-6988-4416-b4a7-249e156f922a" path="/var/lib/kubelet/pods/d99800df-6988-4416-b4a7-249e156f922a/volumes"
Aug 05 17:51:21.097474 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:21.097459    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e62ae463-4a0f-4da0-b10c-950ea6c542fb" path="/var/lib/kubelet/pods/e62ae463-4a0f-4da0-b10c-950ea6c542fb/volumes"
Aug 05 17:51:23.871702 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:23.871667    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"]
Aug 05 17:51:23.872708 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:23.872679    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e62ae463-4a0f-4da0-b10c-950ea6c542fb" containerName="payload-pre-processing"
Aug 05 17:51:23.872708 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:23.872709    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e62ae463-4a0f-4da0-b10c-950ea6c542fb" containerName="payload-pre-processing"
Aug 05 17:51:23.872894 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:23.872736    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d99800df-6988-4416-b4a7-249e156f922a" containerName="maas-api"
Aug 05 17:51:23.872894 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:23.872745    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d99800df-6988-4416-b4a7-249e156f922a" containerName="maas-api"
Aug 05 17:51:23.872894 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:23.872859    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e62ae463-4a0f-4da0-b10c-950ea6c542fb" containerName="payload-pre-processing"
Aug 05 17:51:23.872894 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:23.872869    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d99800df-6988-4416-b4a7-249e156f922a" containerName="maas-api"
Aug 05 17:51:23.877326 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:23.877304    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"
Aug 05 17:51:23.880071 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:23.880053    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ql6kj\""
Aug 05 17:51:23.880198 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:23.880075    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1d0f63d4-serving-cert\""
Aug 05 17:51:23.885958 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:23.885934    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"]
Aug 05 17:51:23.951309 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:23.951276    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppzmd\" (UniqueName: \"kubernetes.io/projected/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-kube-api-access-ppzmd\") pod \"maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm\" (UID: \"5523f9b5-7f47-4e4c-9290-d8adeef8cde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"
Aug 05 17:51:23.951470 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:23.951353    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-maas-api-tls\") pod \"maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm\" (UID: \"5523f9b5-7f47-4e4c-9290-d8adeef8cde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"
Aug 05 17:51:23.951470 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:23.951397    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-openshift-service-ca\") pod \"maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm\" (UID: \"5523f9b5-7f47-4e4c-9290-d8adeef8cde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"
Aug 05 17:51:24.052099 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.052068    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-maas-api-tls\") pod \"maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm\" (UID: \"5523f9b5-7f47-4e4c-9290-d8adeef8cde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"
Aug 05 17:51:24.052099 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.052101    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-openshift-service-ca\") pod \"maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm\" (UID: \"5523f9b5-7f47-4e4c-9290-d8adeef8cde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"
Aug 05 17:51:24.052346 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.052183    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppzmd\" (UniqueName: \"kubernetes.io/projected/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-kube-api-access-ppzmd\") pod \"maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm\" (UID: \"5523f9b5-7f47-4e4c-9290-d8adeef8cde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"
Aug 05 17:51:24.052739 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.052717    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-openshift-service-ca\") pod \"maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm\" (UID: \"5523f9b5-7f47-4e4c-9290-d8adeef8cde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"
Aug 05 17:51:24.054478 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.054456    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-maas-api-tls\") pod \"maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm\" (UID: \"5523f9b5-7f47-4e4c-9290-d8adeef8cde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"
Aug 05 17:51:24.060072 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.060049    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppzmd\" (UniqueName: \"kubernetes.io/projected/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-kube-api-access-ppzmd\") pod \"maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm\" (UID: \"5523f9b5-7f47-4e4c-9290-d8adeef8cde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"
Aug 05 17:51:24.190805 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.190735    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"
Aug 05 17:51:24.332261 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.332196    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"]
Aug 05 17:51:24.335676 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:51:24.335640    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5523f9b5_7f47_4e4c_9290_d8adeef8cde3.slice/crio-b071bcb0b319b6e2ce7358597379f98aabe4c14a14d9cf73be66ebd895e51dd6 WatchSource:0}: Error finding container b071bcb0b319b6e2ce7358597379f98aabe4c14a14d9cf73be66ebd895e51dd6: Status 404 returned error can't find the container with id b071bcb0b319b6e2ce7358597379f98aabe4c14a14d9cf73be66ebd895e51dd6
Aug 05 17:51:24.357115 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.357092    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm" event={"ID":"5523f9b5-7f47-4e4c-9290-d8adeef8cde3","Type":"ContainerStarted","Data":"b071bcb0b319b6e2ce7358597379f98aabe4c14a14d9cf73be66ebd895e51dd6"}
Aug 05 17:51:24.479260 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.479224    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"]
Aug 05 17:51:24.492333 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.492307    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"
Aug 05 17:51:24.495125 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.495096    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"]
Aug 05 17:51:24.495415 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.495394    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-1d0f63d4\""
Aug 05 17:51:24.495762 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.495739    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-1d0f63d4-dockercfg-4t2cn\""
Aug 05 17:51:24.556038 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.556006    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35d8c9a8-8060-47a1-9216-d079b7d5df04-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm\" (UID: \"35d8c9a8-8060-47a1-9216-d079b7d5df04\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"
Aug 05 17:51:24.556233 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.556105    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vtjq\" (UniqueName: \"kubernetes.io/projected/35d8c9a8-8060-47a1-9216-d079b7d5df04-kube-api-access-8vtjq\") pod \"payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm\" (UID: \"35d8c9a8-8060-47a1-9216-d079b7d5df04\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"
Aug 05 17:51:24.657292 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.657262    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35d8c9a8-8060-47a1-9216-d079b7d5df04-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm\" (UID: \"35d8c9a8-8060-47a1-9216-d079b7d5df04\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"
Aug 05 17:51:24.657462 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.657327    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vtjq\" (UniqueName: \"kubernetes.io/projected/35d8c9a8-8060-47a1-9216-d079b7d5df04-kube-api-access-8vtjq\") pod \"payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm\" (UID: \"35d8c9a8-8060-47a1-9216-d079b7d5df04\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"
Aug 05 17:51:24.657984 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.657963    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35d8c9a8-8060-47a1-9216-d079b7d5df04-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm\" (UID: \"35d8c9a8-8060-47a1-9216-d079b7d5df04\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"
Aug 05 17:51:24.665204 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.665179    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vtjq\" (UniqueName: \"kubernetes.io/projected/35d8c9a8-8060-47a1-9216-d079b7d5df04-kube-api-access-8vtjq\") pod \"payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm\" (UID: \"35d8c9a8-8060-47a1-9216-d079b7d5df04\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"
Aug 05 17:51:24.806188 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.806166    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"
Aug 05 17:51:24.938793 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:24.938764    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"]
Aug 05 17:51:24.940143 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:51:24.940116    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35d8c9a8_8060_47a1_9216_d079b7d5df04.slice/crio-8b376a6ccf37f8da2f634106be902564cfb954a227024520d7dea0de7b284757 WatchSource:0}: Error finding container 8b376a6ccf37f8da2f634106be902564cfb954a227024520d7dea0de7b284757: Status 404 returned error can't find the container with id 8b376a6ccf37f8da2f634106be902564cfb954a227024520d7dea0de7b284757
Aug 05 17:51:25.363656 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:25.363617    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm" event={"ID":"5523f9b5-7f47-4e4c-9290-d8adeef8cde3","Type":"ContainerStarted","Data":"002368a1fc6775421568baaebed5a902c6b6b844f9f83894cfb4752547b3f137"}
Aug 05 17:51:25.363843 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:25.363709    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"
Aug 05 17:51:25.365114 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:25.365092    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm" event={"ID":"35d8c9a8-8060-47a1-9216-d079b7d5df04","Type":"ContainerStarted","Data":"dc3a9cad21c4e5e5e793309abc2dfdff1383bd1eca37236d67def7b164887ae0"}
Aug 05 17:51:25.365240 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:25.365127    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm" event={"ID":"35d8c9a8-8060-47a1-9216-d079b7d5df04","Type":"ContainerStarted","Data":"8b376a6ccf37f8da2f634106be902564cfb954a227024520d7dea0de7b284757"}
Aug 05 17:51:25.365240 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:25.365183    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"
Aug 05 17:51:25.380678 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:25.380637    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm" podStartSLOduration=1.95267994 podStartE2EDuration="2.380625562s" totalImagesPullingTime="427.945622ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:51:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:51:25.379016475 +0000 UTC m=+1898.797331260" watchObservedRunningTime="2026-08-05 17:51:25.380625562 +0000 UTC m=+1898.798940351"
Aug 05 17:51:25.395531 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:25.395483    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm" podStartSLOduration=1.395470159 podStartE2EDuration="1.395470159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:51:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:51:25.393945868 +0000 UTC m=+1898.812260628" watchObservedRunningTime="2026-08-05 17:51:25.395470159 +0000 UTC m=+1898.813784936"
Aug 05 17:51:31.375440 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:31.375409    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"
Aug 05 17:51:36.372285 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:36.372253    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"
Aug 05 17:51:54.259250 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.258952    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"]
Aug 05 17:51:54.260360 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.259531    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm" podUID="5523f9b5-7f47-4e4c-9290-d8adeef8cde3" containerName="maas-api" containerID="cri-o://002368a1fc6775421568baaebed5a902c6b6b844f9f83894cfb4752547b3f137" gracePeriod=30
Aug 05 17:51:54.321076 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.321044    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"]
Aug 05 17:51:54.322383 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.321306    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm" podUID="35d8c9a8-8060-47a1-9216-d079b7d5df04" containerName="payload-pre-processing" containerID="cri-o://dc3a9cad21c4e5e5e793309abc2dfdff1383bd1eca37236d67def7b164887ae0" gracePeriod=30
Aug 05 17:51:54.502711 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.502677    2556 generic.go:356] "Generic (PLEG): container finished" podID="35d8c9a8-8060-47a1-9216-d079b7d5df04" containerID="dc3a9cad21c4e5e5e793309abc2dfdff1383bd1eca37236d67def7b164887ae0" exitCode=0
Aug 05 17:51:54.502920 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.502787    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm" event={"ID":"35d8c9a8-8060-47a1-9216-d079b7d5df04","Type":"ContainerDied","Data":"dc3a9cad21c4e5e5e793309abc2dfdff1383bd1eca37236d67def7b164887ae0"}
Aug 05 17:51:54.504423 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.504399    2556 generic.go:356] "Generic (PLEG): container finished" podID="5523f9b5-7f47-4e4c-9290-d8adeef8cde3" containerID="002368a1fc6775421568baaebed5a902c6b6b844f9f83894cfb4752547b3f137" exitCode=0
Aug 05 17:51:54.504525 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.504434    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm" event={"ID":"5523f9b5-7f47-4e4c-9290-d8adeef8cde3","Type":"ContainerDied","Data":"002368a1fc6775421568baaebed5a902c6b6b844f9f83894cfb4752547b3f137"}
Aug 05 17:51:54.574842 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.574819    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"
Aug 05 17:51:54.596042 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.595984    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"
Aug 05 17:51:54.700636 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.700596    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35d8c9a8-8060-47a1-9216-d079b7d5df04-plugins-config-volume\") pod \"35d8c9a8-8060-47a1-9216-d079b7d5df04\" (UID: \"35d8c9a8-8060-47a1-9216-d079b7d5df04\") "
Aug 05 17:51:54.700794 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.700671    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vtjq\" (UniqueName: \"kubernetes.io/projected/35d8c9a8-8060-47a1-9216-d079b7d5df04-kube-api-access-8vtjq\") pod \"35d8c9a8-8060-47a1-9216-d079b7d5df04\" (UID: \"35d8c9a8-8060-47a1-9216-d079b7d5df04\") "
Aug 05 17:51:54.700794 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.700724    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-maas-api-tls\") pod \"5523f9b5-7f47-4e4c-9290-d8adeef8cde3\" (UID: \"5523f9b5-7f47-4e4c-9290-d8adeef8cde3\") "
Aug 05 17:51:54.700794 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.700773    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-openshift-service-ca\") pod \"5523f9b5-7f47-4e4c-9290-d8adeef8cde3\" (UID: \"5523f9b5-7f47-4e4c-9290-d8adeef8cde3\") "
Aug 05 17:51:54.700965 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.700804    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppzmd\" (UniqueName: \"kubernetes.io/projected/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-kube-api-access-ppzmd\") pod \"5523f9b5-7f47-4e4c-9290-d8adeef8cde3\" (UID: \"5523f9b5-7f47-4e4c-9290-d8adeef8cde3\") "
Aug 05 17:51:54.701100 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.701077    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35d8c9a8-8060-47a1-9216-d079b7d5df04-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "35d8c9a8-8060-47a1-9216-d079b7d5df04" (UID: "35d8c9a8-8060-47a1-9216-d079b7d5df04"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:51:54.701190 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.701139    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5523f9b5-7f47-4e4c-9290-d8adeef8cde3" (UID: "5523f9b5-7f47-4e4c-9290-d8adeef8cde3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:51:54.702848 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.702821    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-kube-api-access-ppzmd" (OuterVolumeSpecName: "kube-api-access-ppzmd") pod "5523f9b5-7f47-4e4c-9290-d8adeef8cde3" (UID: "5523f9b5-7f47-4e4c-9290-d8adeef8cde3"). InnerVolumeSpecName "kube-api-access-ppzmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:51:54.702917 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.702852    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35d8c9a8-8060-47a1-9216-d079b7d5df04-kube-api-access-8vtjq" (OuterVolumeSpecName: "kube-api-access-8vtjq") pod "35d8c9a8-8060-47a1-9216-d079b7d5df04" (UID: "35d8c9a8-8060-47a1-9216-d079b7d5df04"). InnerVolumeSpecName "kube-api-access-8vtjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:51:54.702917 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.702873    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5523f9b5-7f47-4e4c-9290-d8adeef8cde3" (UID: "5523f9b5-7f47-4e4c-9290-d8adeef8cde3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:51:54.802133 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.802078    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35d8c9a8-8060-47a1-9216-d079b7d5df04-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:51:54.802133 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.802099    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vtjq\" (UniqueName: \"kubernetes.io/projected/35d8c9a8-8060-47a1-9216-d079b7d5df04-kube-api-access-8vtjq\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:51:54.802133 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.802110    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:51:54.802133 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.802118    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:51:54.802133 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:54.802126    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppzmd\" (UniqueName: \"kubernetes.io/projected/5523f9b5-7f47-4e4c-9290-d8adeef8cde3-kube-api-access-ppzmd\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:51:55.511085 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:55.511056    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"
Aug 05 17:51:55.511571 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:55.511079    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm" event={"ID":"35d8c9a8-8060-47a1-9216-d079b7d5df04","Type":"ContainerDied","Data":"8b376a6ccf37f8da2f634106be902564cfb954a227024520d7dea0de7b284757"}
Aug 05 17:51:55.511571 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:55.511179    2556 scope.go:117] "RemoveContainer" containerID="dc3a9cad21c4e5e5e793309abc2dfdff1383bd1eca37236d67def7b164887ae0"
Aug 05 17:51:55.512919 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:55.512900    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"
Aug 05 17:51:55.513214 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:55.512903    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm" event={"ID":"5523f9b5-7f47-4e4c-9290-d8adeef8cde3","Type":"ContainerDied","Data":"b071bcb0b319b6e2ce7358597379f98aabe4c14a14d9cf73be66ebd895e51dd6"}
Aug 05 17:51:55.520135 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:55.520119    2556 scope.go:117] "RemoveContainer" containerID="002368a1fc6775421568baaebed5a902c6b6b844f9f83894cfb4752547b3f137"
Aug 05 17:51:55.529260 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:55.529237    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"]
Aug 05 17:51:55.532090 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:55.532060    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d0f63d4-6dd444d59b-h9lvm"]
Aug 05 17:51:55.542619 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:55.542591    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"]
Aug 05 17:51:55.546061 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:55.546042    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d0f63d4-64b8b6cd48-wmnfm"]
Aug 05 17:51:57.097330 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:57.097290    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35d8c9a8-8060-47a1-9216-d079b7d5df04" path="/var/lib/kubelet/pods/35d8c9a8-8060-47a1-9216-d079b7d5df04/volumes"
Aug 05 17:51:57.097859 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:51:57.097838    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5523f9b5-7f47-4e4c-9290-d8adeef8cde3" path="/var/lib/kubelet/pods/5523f9b5-7f47-4e4c-9290-d8adeef8cde3/volumes"
Aug 05 17:52:02.886956 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:02.886924    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"]
Aug 05 17:52:02.887450 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:02.887434    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5523f9b5-7f47-4e4c-9290-d8adeef8cde3" containerName="maas-api"
Aug 05 17:52:02.887500 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:02.887453    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5523f9b5-7f47-4e4c-9290-d8adeef8cde3" containerName="maas-api"
Aug 05 17:52:02.887500 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:02.887475    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35d8c9a8-8060-47a1-9216-d079b7d5df04" containerName="payload-pre-processing"
Aug 05 17:52:02.887500 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:02.887484    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="35d8c9a8-8060-47a1-9216-d079b7d5df04" containerName="payload-pre-processing"
Aug 05 17:52:02.887600 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:02.887568    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="35d8c9a8-8060-47a1-9216-d079b7d5df04" containerName="payload-pre-processing"
Aug 05 17:52:02.887600 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:02.887581    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5523f9b5-7f47-4e4c-9290-d8adeef8cde3" containerName="maas-api"
Aug 05 17:52:02.890547 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:02.890530    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"
Aug 05 17:52:02.893229 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:02.893204    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ql6kj\""
Aug 05 17:52:02.893358 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:02.893274    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-12e4a979-serving-cert\""
Aug 05 17:52:02.900252 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:02.900233    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"]
Aug 05 17:52:02.967771 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:02.967741    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f52a89b-3785-4e16-b497-7c3a3acd44fb-maas-api-tls\") pod \"maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb\" (UID: \"4f52a89b-3785-4e16-b497-7c3a3acd44fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"
Aug 05 17:52:02.967942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:02.967812    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f52a89b-3785-4e16-b497-7c3a3acd44fb-openshift-service-ca\") pod \"maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb\" (UID: \"4f52a89b-3785-4e16-b497-7c3a3acd44fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"
Aug 05 17:52:02.967942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:02.967847    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqgh4\" (UniqueName: \"kubernetes.io/projected/4f52a89b-3785-4e16-b497-7c3a3acd44fb-kube-api-access-lqgh4\") pod \"maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb\" (UID: \"4f52a89b-3785-4e16-b497-7c3a3acd44fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"
Aug 05 17:52:03.068702 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.068674    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f52a89b-3785-4e16-b497-7c3a3acd44fb-openshift-service-ca\") pod \"maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb\" (UID: \"4f52a89b-3785-4e16-b497-7c3a3acd44fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"
Aug 05 17:52:03.068859 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.068709    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqgh4\" (UniqueName: \"kubernetes.io/projected/4f52a89b-3785-4e16-b497-7c3a3acd44fb-kube-api-access-lqgh4\") pod \"maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb\" (UID: \"4f52a89b-3785-4e16-b497-7c3a3acd44fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"
Aug 05 17:52:03.068859 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.068765    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f52a89b-3785-4e16-b497-7c3a3acd44fb-maas-api-tls\") pod \"maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb\" (UID: \"4f52a89b-3785-4e16-b497-7c3a3acd44fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"
Aug 05 17:52:03.069366 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.069347    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f52a89b-3785-4e16-b497-7c3a3acd44fb-openshift-service-ca\") pod \"maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb\" (UID: \"4f52a89b-3785-4e16-b497-7c3a3acd44fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"
Aug 05 17:52:03.071138 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.071114    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f52a89b-3785-4e16-b497-7c3a3acd44fb-maas-api-tls\") pod \"maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb\" (UID: \"4f52a89b-3785-4e16-b497-7c3a3acd44fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"
Aug 05 17:52:03.076643 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.076624    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqgh4\" (UniqueName: \"kubernetes.io/projected/4f52a89b-3785-4e16-b497-7c3a3acd44fb-kube-api-access-lqgh4\") pod \"maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb\" (UID: \"4f52a89b-3785-4e16-b497-7c3a3acd44fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"
Aug 05 17:52:03.203822 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.203752    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"
Aug 05 17:52:03.340800 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.340711    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"]
Aug 05 17:52:03.343088 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:52:03.343062    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f52a89b_3785_4e16_b497_7c3a3acd44fb.slice/crio-d6e3d4dfa698ecd385a7bfab43c46af20cce673b1a49d72e9ee5d39c87a0366e WatchSource:0}: Error finding container d6e3d4dfa698ecd385a7bfab43c46af20cce673b1a49d72e9ee5d39c87a0366e: Status 404 returned error can't find the container with id d6e3d4dfa698ecd385a7bfab43c46af20cce673b1a49d72e9ee5d39c87a0366e
Aug 05 17:52:03.552055 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.552015    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb" event={"ID":"4f52a89b-3785-4e16-b497-7c3a3acd44fb","Type":"ContainerStarted","Data":"d6e3d4dfa698ecd385a7bfab43c46af20cce673b1a49d72e9ee5d39c87a0366e"}
Aug 05 17:52:03.650103 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.650065    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"]
Aug 05 17:52:03.654518 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.654496    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"
Aug 05 17:52:03.657095 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.657076    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-12e4a979\""
Aug 05 17:52:03.657210 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.657104    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-12e4a979-dockercfg-hk6k4\""
Aug 05 17:52:03.664696 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.664644    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"]
Aug 05 17:52:03.732135 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.732112    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"]
Aug 05 17:52:03.736643 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.736623    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"
Aug 05 17:52:03.746855 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.746831    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"]
Aug 05 17:52:03.776333 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.776308    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2904855c-b0e4-480a-a144-f55c59ace29a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n\" (UID: \"2904855c-b0e4-480a-a144-f55c59ace29a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"
Aug 05 17:52:03.776459 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.776347    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9z4t\" (UniqueName: \"kubernetes.io/projected/2904855c-b0e4-480a-a144-f55c59ace29a-kube-api-access-c9z4t\") pod \"payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n\" (UID: \"2904855c-b0e4-480a-a144-f55c59ace29a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"
Aug 05 17:52:03.877267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.877181    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9z4t\" (UniqueName: \"kubernetes.io/projected/2904855c-b0e4-480a-a144-f55c59ace29a-kube-api-access-c9z4t\") pod \"payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n\" (UID: \"2904855c-b0e4-480a-a144-f55c59ace29a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"
Aug 05 17:52:03.877267 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.877254    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4268e69-2d2c-419e-8d9f-5288ba9a24cb-plugins-config-volume\") pod \"payload-processing-e2e-derive-12e4a979-f69b896db-96d9h\" (UID: \"d4268e69-2d2c-419e-8d9f-5288ba9a24cb\") " pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"
Aug 05 17:52:03.877494 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.877403    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25cc9\" (UniqueName: \"kubernetes.io/projected/d4268e69-2d2c-419e-8d9f-5288ba9a24cb-kube-api-access-25cc9\") pod \"payload-processing-e2e-derive-12e4a979-f69b896db-96d9h\" (UID: \"d4268e69-2d2c-419e-8d9f-5288ba9a24cb\") " pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"
Aug 05 17:52:03.877494 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.877448    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2904855c-b0e4-480a-a144-f55c59ace29a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n\" (UID: \"2904855c-b0e4-480a-a144-f55c59ace29a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"
Aug 05 17:52:03.878166 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.878125    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2904855c-b0e4-480a-a144-f55c59ace29a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n\" (UID: \"2904855c-b0e4-480a-a144-f55c59ace29a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"
Aug 05 17:52:03.885646 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.885618    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9z4t\" (UniqueName: \"kubernetes.io/projected/2904855c-b0e4-480a-a144-f55c59ace29a-kube-api-access-c9z4t\") pod \"payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n\" (UID: \"2904855c-b0e4-480a-a144-f55c59ace29a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"
Aug 05 17:52:03.966507 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.966476    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"
Aug 05 17:52:03.978824 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.978800    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25cc9\" (UniqueName: \"kubernetes.io/projected/d4268e69-2d2c-419e-8d9f-5288ba9a24cb-kube-api-access-25cc9\") pod \"payload-processing-e2e-derive-12e4a979-f69b896db-96d9h\" (UID: \"d4268e69-2d2c-419e-8d9f-5288ba9a24cb\") " pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"
Aug 05 17:52:03.978906 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.978845    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4268e69-2d2c-419e-8d9f-5288ba9a24cb-plugins-config-volume\") pod \"payload-processing-e2e-derive-12e4a979-f69b896db-96d9h\" (UID: \"d4268e69-2d2c-419e-8d9f-5288ba9a24cb\") " pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"
Aug 05 17:52:03.979477 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.979458    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4268e69-2d2c-419e-8d9f-5288ba9a24cb-plugins-config-volume\") pod \"payload-processing-e2e-derive-12e4a979-f69b896db-96d9h\" (UID: \"d4268e69-2d2c-419e-8d9f-5288ba9a24cb\") " pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"
Aug 05 17:52:03.986743 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:03.986722    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25cc9\" (UniqueName: \"kubernetes.io/projected/d4268e69-2d2c-419e-8d9f-5288ba9a24cb-kube-api-access-25cc9\") pod \"payload-processing-e2e-derive-12e4a979-f69b896db-96d9h\" (UID: \"d4268e69-2d2c-419e-8d9f-5288ba9a24cb\") " pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"
Aug 05 17:52:04.050052 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:04.049936    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"
Aug 05 17:52:04.102305 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:04.102278    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"]
Aug 05 17:52:04.104448 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:52:04.104406    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2904855c_b0e4_480a_a144_f55c59ace29a.slice/crio-5106986b01609c9317ea87e552e105447a5dfa47b8e516114f5bd463e425b90a WatchSource:0}: Error finding container 5106986b01609c9317ea87e552e105447a5dfa47b8e516114f5bd463e425b90a: Status 404 returned error can't find the container with id 5106986b01609c9317ea87e552e105447a5dfa47b8e516114f5bd463e425b90a
Aug 05 17:52:04.205301 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:04.205278    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"]
Aug 05 17:52:04.207301 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:52:04.207259    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4268e69_2d2c_419e_8d9f_5288ba9a24cb.slice/crio-954d3d12f89e4e5c2bd3b3ece82c452a7af32da1c8b5483a8959fb75e2f77f9e WatchSource:0}: Error finding container 954d3d12f89e4e5c2bd3b3ece82c452a7af32da1c8b5483a8959fb75e2f77f9e: Status 404 returned error can't find the container with id 954d3d12f89e4e5c2bd3b3ece82c452a7af32da1c8b5483a8959fb75e2f77f9e
Aug 05 17:52:04.559076 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:04.559033    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n" event={"ID":"2904855c-b0e4-480a-a144-f55c59ace29a","Type":"ContainerStarted","Data":"e0af4b62389829576ab03e27f69fa055c3c7f2bf63d544b05faedc261fdb922e"}
Aug 05 17:52:04.559266 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:04.559103    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n" event={"ID":"2904855c-b0e4-480a-a144-f55c59ace29a","Type":"ContainerStarted","Data":"5106986b01609c9317ea87e552e105447a5dfa47b8e516114f5bd463e425b90a"}
Aug 05 17:52:04.559266 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:04.559177    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"
Aug 05 17:52:04.560674 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:04.560649    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb" event={"ID":"4f52a89b-3785-4e16-b497-7c3a3acd44fb","Type":"ContainerStarted","Data":"33eff9512cfeb5fcdc9b8750af2628009ebef4da496df8067534329e8eff3a17"}
Aug 05 17:52:04.560787 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:04.560708    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"
Aug 05 17:52:04.562311 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:04.562288    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h" event={"ID":"d4268e69-2d2c-419e-8d9f-5288ba9a24cb","Type":"ContainerStarted","Data":"ce4016a417037f9959f655dbfd627c4a194a9592e92598a999d4ed29572cf614"}
Aug 05 17:52:04.562426 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:04.562330    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h" event={"ID":"d4268e69-2d2c-419e-8d9f-5288ba9a24cb","Type":"ContainerStarted","Data":"954d3d12f89e4e5c2bd3b3ece82c452a7af32da1c8b5483a8959fb75e2f77f9e"}
Aug 05 17:52:04.562426 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:04.562408    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"
Aug 05 17:52:04.575401 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:04.575363    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n" podStartSLOduration=1.575350874 podStartE2EDuration="1.575350874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:52:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:52:04.573686892 +0000 UTC m=+1937.992001650" watchObservedRunningTime="2026-08-05 17:52:04.575350874 +0000 UTC m=+1937.993665632"
Aug 05 17:52:04.589625 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:04.589588    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h" podStartSLOduration=1.589573974 podStartE2EDuration="1.589573974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:52:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:52:04.587363869 +0000 UTC m=+1938.005678646" watchObservedRunningTime="2026-08-05 17:52:04.589573974 +0000 UTC m=+1938.007888730"
Aug 05 17:52:04.603349 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:04.603306    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb" podStartSLOduration=2.245394045 podStartE2EDuration="2.603293314s" totalImagesPullingTime="357.899269ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:52:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:52:04.602224742 +0000 UTC m=+1938.020539527" watchObservedRunningTime="2026-08-05 17:52:04.603293314 +0000 UTC m=+1938.021608071"
Aug 05 17:52:10.572573 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:10.572544    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"
Aug 05 17:52:15.568290 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:15.568258    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"
Aug 05 17:52:15.568704 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:15.568342    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"
Aug 05 17:52:43.376924 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.376838    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"]
Aug 05 17:52:43.377500 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.377230    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb" podUID="4f52a89b-3785-4e16-b497-7c3a3acd44fb" containerName="maas-api" containerID="cri-o://33eff9512cfeb5fcdc9b8750af2628009ebef4da496df8067534329e8eff3a17" gracePeriod=30
Aug 05 17:52:43.428057 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.428030    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"]
Aug 05 17:52:43.428257 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.428235    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h" podUID="d4268e69-2d2c-419e-8d9f-5288ba9a24cb" containerName="payload-processing" containerID="cri-o://ce4016a417037f9959f655dbfd627c4a194a9592e92598a999d4ed29572cf614" gracePeriod=30
Aug 05 17:52:43.435479 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.435456    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"]
Aug 05 17:52:43.435660 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.435641    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n" podUID="2904855c-b0e4-480a-a144-f55c59ace29a" containerName="payload-pre-processing" containerID="cri-o://e0af4b62389829576ab03e27f69fa055c3c7f2bf63d544b05faedc261fdb922e" gracePeriod=30
Aug 05 17:52:43.695431 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.694823    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"
Aug 05 17:52:43.725138 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.725107    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"
Aug 05 17:52:43.750861 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.750835    2556 generic.go:356] "Generic (PLEG): container finished" podID="2904855c-b0e4-480a-a144-f55c59ace29a" containerID="e0af4b62389829576ab03e27f69fa055c3c7f2bf63d544b05faedc261fdb922e" exitCode=0
Aug 05 17:52:43.750970 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.750907    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n" event={"ID":"2904855c-b0e4-480a-a144-f55c59ace29a","Type":"ContainerDied","Data":"e0af4b62389829576ab03e27f69fa055c3c7f2bf63d544b05faedc261fdb922e"}
Aug 05 17:52:43.752353 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.752326    2556 generic.go:356] "Generic (PLEG): container finished" podID="4f52a89b-3785-4e16-b497-7c3a3acd44fb" containerID="33eff9512cfeb5fcdc9b8750af2628009ebef4da496df8067534329e8eff3a17" exitCode=0
Aug 05 17:52:43.752480 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.752372    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"
Aug 05 17:52:43.752480 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.752402    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb" event={"ID":"4f52a89b-3785-4e16-b497-7c3a3acd44fb","Type":"ContainerDied","Data":"33eff9512cfeb5fcdc9b8750af2628009ebef4da496df8067534329e8eff3a17"}
Aug 05 17:52:43.752480 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.752452    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb" event={"ID":"4f52a89b-3785-4e16-b497-7c3a3acd44fb","Type":"ContainerDied","Data":"d6e3d4dfa698ecd385a7bfab43c46af20cce673b1a49d72e9ee5d39c87a0366e"}
Aug 05 17:52:43.752624 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.752486    2556 scope.go:117] "RemoveContainer" containerID="33eff9512cfeb5fcdc9b8750af2628009ebef4da496df8067534329e8eff3a17"
Aug 05 17:52:43.753598 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.753575    2556 generic.go:356] "Generic (PLEG): container finished" podID="d4268e69-2d2c-419e-8d9f-5288ba9a24cb" containerID="ce4016a417037f9959f655dbfd627c4a194a9592e92598a999d4ed29572cf614" exitCode=0
Aug 05 17:52:43.753657 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.753610    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h" event={"ID":"d4268e69-2d2c-419e-8d9f-5288ba9a24cb","Type":"ContainerDied","Data":"ce4016a417037f9959f655dbfd627c4a194a9592e92598a999d4ed29572cf614"}
Aug 05 17:52:43.753657 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.753648    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h" event={"ID":"d4268e69-2d2c-419e-8d9f-5288ba9a24cb","Type":"ContainerDied","Data":"954d3d12f89e4e5c2bd3b3ece82c452a7af32da1c8b5483a8959fb75e2f77f9e"}
Aug 05 17:52:43.753725 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.753675    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"
Aug 05 17:52:43.755710 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.755695    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"
Aug 05 17:52:43.762294 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.762277    2556 scope.go:117] "RemoveContainer" containerID="33eff9512cfeb5fcdc9b8750af2628009ebef4da496df8067534329e8eff3a17"
Aug 05 17:52:43.762545 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:52:43.762529    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33eff9512cfeb5fcdc9b8750af2628009ebef4da496df8067534329e8eff3a17\": container with ID starting with 33eff9512cfeb5fcdc9b8750af2628009ebef4da496df8067534329e8eff3a17 not found: ID does not exist" containerID="33eff9512cfeb5fcdc9b8750af2628009ebef4da496df8067534329e8eff3a17"
Aug 05 17:52:43.762589 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.762553    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33eff9512cfeb5fcdc9b8750af2628009ebef4da496df8067534329e8eff3a17"} err="failed to get container status \"33eff9512cfeb5fcdc9b8750af2628009ebef4da496df8067534329e8eff3a17\": rpc error: code = NotFound desc = could not find container \"33eff9512cfeb5fcdc9b8750af2628009ebef4da496df8067534329e8eff3a17\": container with ID starting with 33eff9512cfeb5fcdc9b8750af2628009ebef4da496df8067534329e8eff3a17 not found: ID does not exist"
Aug 05 17:52:43.762589 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.762567    2556 scope.go:117] "RemoveContainer" containerID="ce4016a417037f9959f655dbfd627c4a194a9592e92598a999d4ed29572cf614"
Aug 05 17:52:43.771431 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.771389    2556 scope.go:117] "RemoveContainer" containerID="ce4016a417037f9959f655dbfd627c4a194a9592e92598a999d4ed29572cf614"
Aug 05 17:52:43.771821 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:52:43.771734    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce4016a417037f9959f655dbfd627c4a194a9592e92598a999d4ed29572cf614\": container with ID starting with ce4016a417037f9959f655dbfd627c4a194a9592e92598a999d4ed29572cf614 not found: ID does not exist" containerID="ce4016a417037f9959f655dbfd627c4a194a9592e92598a999d4ed29572cf614"
Aug 05 17:52:43.771916 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.771844    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce4016a417037f9959f655dbfd627c4a194a9592e92598a999d4ed29572cf614"} err="failed to get container status \"ce4016a417037f9959f655dbfd627c4a194a9592e92598a999d4ed29572cf614\": rpc error: code = NotFound desc = could not find container \"ce4016a417037f9959f655dbfd627c4a194a9592e92598a999d4ed29572cf614\": container with ID starting with ce4016a417037f9959f655dbfd627c4a194a9592e92598a999d4ed29572cf614 not found: ID does not exist"
Aug 05 17:52:43.806392 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.806368    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f52a89b-3785-4e16-b497-7c3a3acd44fb-maas-api-tls\") pod \"4f52a89b-3785-4e16-b497-7c3a3acd44fb\" (UID: \"4f52a89b-3785-4e16-b497-7c3a3acd44fb\") "
Aug 05 17:52:43.806509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.806415    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2904855c-b0e4-480a-a144-f55c59ace29a-plugins-config-volume\") pod \"2904855c-b0e4-480a-a144-f55c59ace29a\" (UID: \"2904855c-b0e4-480a-a144-f55c59ace29a\") "
Aug 05 17:52:43.806509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.806439    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9z4t\" (UniqueName: \"kubernetes.io/projected/2904855c-b0e4-480a-a144-f55c59ace29a-kube-api-access-c9z4t\") pod \"2904855c-b0e4-480a-a144-f55c59ace29a\" (UID: \"2904855c-b0e4-480a-a144-f55c59ace29a\") "
Aug 05 17:52:43.806509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.806470    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4268e69-2d2c-419e-8d9f-5288ba9a24cb-plugins-config-volume\") pod \"d4268e69-2d2c-419e-8d9f-5288ba9a24cb\" (UID: \"d4268e69-2d2c-419e-8d9f-5288ba9a24cb\") "
Aug 05 17:52:43.806509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.806489    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqgh4\" (UniqueName: \"kubernetes.io/projected/4f52a89b-3785-4e16-b497-7c3a3acd44fb-kube-api-access-lqgh4\") pod \"4f52a89b-3785-4e16-b497-7c3a3acd44fb\" (UID: \"4f52a89b-3785-4e16-b497-7c3a3acd44fb\") "
Aug 05 17:52:43.806509 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.806509    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25cc9\" (UniqueName: \"kubernetes.io/projected/d4268e69-2d2c-419e-8d9f-5288ba9a24cb-kube-api-access-25cc9\") pod \"d4268e69-2d2c-419e-8d9f-5288ba9a24cb\" (UID: \"d4268e69-2d2c-419e-8d9f-5288ba9a24cb\") "
Aug 05 17:52:43.806775 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.806544    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f52a89b-3785-4e16-b497-7c3a3acd44fb-openshift-service-ca\") pod \"4f52a89b-3785-4e16-b497-7c3a3acd44fb\" (UID: \"4f52a89b-3785-4e16-b497-7c3a3acd44fb\") "
Aug 05 17:52:43.806883 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.806858    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4268e69-2d2c-419e-8d9f-5288ba9a24cb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4268e69-2d2c-419e-8d9f-5288ba9a24cb" (UID: "d4268e69-2d2c-419e-8d9f-5288ba9a24cb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:52:43.807077 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.807051    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f52a89b-3785-4e16-b497-7c3a3acd44fb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4f52a89b-3785-4e16-b497-7c3a3acd44fb" (UID: "4f52a89b-3785-4e16-b497-7c3a3acd44fb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:52:43.807202 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.807175    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2904855c-b0e4-480a-a144-f55c59ace29a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2904855c-b0e4-480a-a144-f55c59ace29a" (UID: "2904855c-b0e4-480a-a144-f55c59ace29a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:52:43.809280 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.809167    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2904855c-b0e4-480a-a144-f55c59ace29a-kube-api-access-c9z4t" (OuterVolumeSpecName: "kube-api-access-c9z4t") pod "2904855c-b0e4-480a-a144-f55c59ace29a" (UID: "2904855c-b0e4-480a-a144-f55c59ace29a"). InnerVolumeSpecName "kube-api-access-c9z4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:52:43.809280 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.809143    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4268e69-2d2c-419e-8d9f-5288ba9a24cb-kube-api-access-25cc9" (OuterVolumeSpecName: "kube-api-access-25cc9") pod "d4268e69-2d2c-419e-8d9f-5288ba9a24cb" (UID: "d4268e69-2d2c-419e-8d9f-5288ba9a24cb"). InnerVolumeSpecName "kube-api-access-25cc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:52:43.809280 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.809249    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f52a89b-3785-4e16-b497-7c3a3acd44fb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4f52a89b-3785-4e16-b497-7c3a3acd44fb" (UID: "4f52a89b-3785-4e16-b497-7c3a3acd44fb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:52:43.809280 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.809255    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f52a89b-3785-4e16-b497-7c3a3acd44fb-kube-api-access-lqgh4" (OuterVolumeSpecName: "kube-api-access-lqgh4") pod "4f52a89b-3785-4e16-b497-7c3a3acd44fb" (UID: "4f52a89b-3785-4e16-b497-7c3a3acd44fb"). InnerVolumeSpecName "kube-api-access-lqgh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:52:43.908106 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.908025    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f52a89b-3785-4e16-b497-7c3a3acd44fb-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:52:43.908106 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.908061    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f52a89b-3785-4e16-b497-7c3a3acd44fb-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:52:43.908106 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.908075    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2904855c-b0e4-480a-a144-f55c59ace29a-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:52:43.908106 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.908087    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9z4t\" (UniqueName: \"kubernetes.io/projected/2904855c-b0e4-480a-a144-f55c59ace29a-kube-api-access-c9z4t\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:52:43.908106 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.908101    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4268e69-2d2c-419e-8d9f-5288ba9a24cb-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:52:43.908398 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.908115    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqgh4\" (UniqueName: \"kubernetes.io/projected/4f52a89b-3785-4e16-b497-7c3a3acd44fb-kube-api-access-lqgh4\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:52:43.908398 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:43.908128    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25cc9\" (UniqueName: \"kubernetes.io/projected/d4268e69-2d2c-419e-8d9f-5288ba9a24cb-kube-api-access-25cc9\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:52:44.075607 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:44.075583    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"]
Aug 05 17:52:44.078778 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:44.078759    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-12e4a979-f69b896db-96d9h"]
Aug 05 17:52:44.088448 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:44.088420    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"]
Aug 05 17:52:44.092402 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:44.092379    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-12e4a979-6d7dfdfb49-mkfxb"]
Aug 05 17:52:44.762097 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:44.762066    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"
Aug 05 17:52:44.762097 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:44.762075    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n" event={"ID":"2904855c-b0e4-480a-a144-f55c59ace29a","Type":"ContainerDied","Data":"5106986b01609c9317ea87e552e105447a5dfa47b8e516114f5bd463e425b90a"}
Aug 05 17:52:44.762580 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:44.762181    2556 scope.go:117] "RemoveContainer" containerID="e0af4b62389829576ab03e27f69fa055c3c7f2bf63d544b05faedc261fdb922e"
Aug 05 17:52:44.783068 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:44.783045    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"]
Aug 05 17:52:44.786922 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:44.786901    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-12e4a979-746b9d56dd-2k58n"]
Aug 05 17:52:45.096516 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:45.096440    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2904855c-b0e4-480a-a144-f55c59ace29a" path="/var/lib/kubelet/pods/2904855c-b0e4-480a-a144-f55c59ace29a/volumes"
Aug 05 17:52:45.096823 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:45.096809    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f52a89b-3785-4e16-b497-7c3a3acd44fb" path="/var/lib/kubelet/pods/4f52a89b-3785-4e16-b497-7c3a3acd44fb/volumes"
Aug 05 17:52:45.097120 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:52:45.097109    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4268e69-2d2c-419e-8d9f-5288ba9a24cb" path="/var/lib/kubelet/pods/d4268e69-2d2c-419e-8d9f-5288ba9a24cb/volumes"
Aug 05 17:54:46.252223 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.252109    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"]
Aug 05 17:54:46.252778 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.252686    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f52a89b-3785-4e16-b497-7c3a3acd44fb" containerName="maas-api"
Aug 05 17:54:46.252778 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.252707    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f52a89b-3785-4e16-b497-7c3a3acd44fb" containerName="maas-api"
Aug 05 17:54:46.252778 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.252747    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2904855c-b0e4-480a-a144-f55c59ace29a" containerName="payload-pre-processing"
Aug 05 17:54:46.252778 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.252756    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2904855c-b0e4-480a-a144-f55c59ace29a" containerName="payload-pre-processing"
Aug 05 17:54:46.252778 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.252769    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4268e69-2d2c-419e-8d9f-5288ba9a24cb" containerName="payload-processing"
Aug 05 17:54:46.252778 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.252778    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4268e69-2d2c-419e-8d9f-5288ba9a24cb" containerName="payload-processing"
Aug 05 17:54:46.253101 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.252852    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4268e69-2d2c-419e-8d9f-5288ba9a24cb" containerName="payload-processing"
Aug 05 17:54:46.253101 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.252865    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f52a89b-3785-4e16-b497-7c3a3acd44fb" containerName="maas-api"
Aug 05 17:54:46.253101 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.252873    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2904855c-b0e4-480a-a144-f55c59ace29a" containerName="payload-pre-processing"
Aug 05 17:54:46.256168 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.256130    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"
Aug 05 17:54:46.258999 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.258973    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4f48a67f-serving-cert\""
Aug 05 17:54:46.259640 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.259623    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ql6kj\""
Aug 05 17:54:46.265579 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.265544    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"]
Aug 05 17:54:46.365463 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.365409    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/546fbd81-44a9-4013-9c07-c7b64f3f3590-maas-api-tls\") pod \"maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t\" (UID: \"546fbd81-44a9-4013-9c07-c7b64f3f3590\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"
Aug 05 17:54:46.365463 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.365449    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/546fbd81-44a9-4013-9c07-c7b64f3f3590-openshift-service-ca\") pod \"maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t\" (UID: \"546fbd81-44a9-4013-9c07-c7b64f3f3590\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"
Aug 05 17:54:46.365710 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.365546    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8nnv\" (UniqueName: \"kubernetes.io/projected/546fbd81-44a9-4013-9c07-c7b64f3f3590-kube-api-access-n8nnv\") pod \"maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t\" (UID: \"546fbd81-44a9-4013-9c07-c7b64f3f3590\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"
Aug 05 17:54:46.466584 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.466552    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/546fbd81-44a9-4013-9c07-c7b64f3f3590-maas-api-tls\") pod \"maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t\" (UID: \"546fbd81-44a9-4013-9c07-c7b64f3f3590\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"
Aug 05 17:54:46.466739 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.466594    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/546fbd81-44a9-4013-9c07-c7b64f3f3590-openshift-service-ca\") pod \"maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t\" (UID: \"546fbd81-44a9-4013-9c07-c7b64f3f3590\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"
Aug 05 17:54:46.466739 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.466631    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8nnv\" (UniqueName: \"kubernetes.io/projected/546fbd81-44a9-4013-9c07-c7b64f3f3590-kube-api-access-n8nnv\") pod \"maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t\" (UID: \"546fbd81-44a9-4013-9c07-c7b64f3f3590\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"
Aug 05 17:54:46.467242 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.467216    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/546fbd81-44a9-4013-9c07-c7b64f3f3590-openshift-service-ca\") pod \"maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t\" (UID: \"546fbd81-44a9-4013-9c07-c7b64f3f3590\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"
Aug 05 17:54:46.468963 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.468941    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/546fbd81-44a9-4013-9c07-c7b64f3f3590-maas-api-tls\") pod \"maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t\" (UID: \"546fbd81-44a9-4013-9c07-c7b64f3f3590\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"
Aug 05 17:54:46.483421 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.483395    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8nnv\" (UniqueName: \"kubernetes.io/projected/546fbd81-44a9-4013-9c07-c7b64f3f3590-kube-api-access-n8nnv\") pod \"maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t\" (UID: \"546fbd81-44a9-4013-9c07-c7b64f3f3590\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"
Aug 05 17:54:46.569141 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.569071    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"
Aug 05 17:54:46.915380 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.915353    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"]
Aug 05 17:54:46.917739 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:54:46.917711    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod546fbd81_44a9_4013_9c07_c7b64f3f3590.slice/crio-fb3891c409d40bab9e0464caf96339218171cd3dad44564bc47fecd82c445f7f WatchSource:0}: Error finding container fb3891c409d40bab9e0464caf96339218171cd3dad44564bc47fecd82c445f7f: Status 404 returned error can't find the container with id fb3891c409d40bab9e0464caf96339218171cd3dad44564bc47fecd82c445f7f
Aug 05 17:54:46.958712 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.958681    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"]
Aug 05 17:54:46.963283 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.963266    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"
Aug 05 17:54:46.966016 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.965997    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-4f48a67f\""
Aug 05 17:54:46.966107 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.966020    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-4f48a67f-dockercfg-sgnx6\""
Aug 05 17:54:46.975305 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:46.975284    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"]
Aug 05 17:54:47.003406 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.003383    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"]
Aug 05 17:54:47.006694 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.006677    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"
Aug 05 17:54:47.018942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.018920    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"]
Aug 05 17:54:47.071250 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.071225    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q55d9\" (UniqueName: \"kubernetes.io/projected/4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458-kube-api-access-q55d9\") pod \"payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7\" (UID: \"4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458\") " pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"
Aug 05 17:54:47.071390 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.071308    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/020d9235-45b5-4677-8c56-f65f2d0f06d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk\" (UID: \"020d9235-45b5-4677-8c56-f65f2d0f06d4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"
Aug 05 17:54:47.071390 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.071372    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458-plugins-config-volume\") pod \"payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7\" (UID: \"4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458\") " pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"
Aug 05 17:54:47.071502 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.071412    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mjnj\" (UniqueName: \"kubernetes.io/projected/020d9235-45b5-4677-8c56-f65f2d0f06d4-kube-api-access-2mjnj\") pod \"payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk\" (UID: \"020d9235-45b5-4677-8c56-f65f2d0f06d4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"
Aug 05 17:54:47.172498 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.172424    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q55d9\" (UniqueName: \"kubernetes.io/projected/4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458-kube-api-access-q55d9\") pod \"payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7\" (UID: \"4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458\") " pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"
Aug 05 17:54:47.172498 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.172467    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/020d9235-45b5-4677-8c56-f65f2d0f06d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk\" (UID: \"020d9235-45b5-4677-8c56-f65f2d0f06d4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"
Aug 05 17:54:47.172498 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.172489    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458-plugins-config-volume\") pod \"payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7\" (UID: \"4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458\") " pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"
Aug 05 17:54:47.172784 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.172524    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mjnj\" (UniqueName: \"kubernetes.io/projected/020d9235-45b5-4677-8c56-f65f2d0f06d4-kube-api-access-2mjnj\") pod \"payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk\" (UID: \"020d9235-45b5-4677-8c56-f65f2d0f06d4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"
Aug 05 17:54:47.173123 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.173100    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/020d9235-45b5-4677-8c56-f65f2d0f06d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk\" (UID: \"020d9235-45b5-4677-8c56-f65f2d0f06d4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"
Aug 05 17:54:47.173252 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.173217    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458-plugins-config-volume\") pod \"payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7\" (UID: \"4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458\") " pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"
Aug 05 17:54:47.180931 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.180909    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mjnj\" (UniqueName: \"kubernetes.io/projected/020d9235-45b5-4677-8c56-f65f2d0f06d4-kube-api-access-2mjnj\") pod \"payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk\" (UID: \"020d9235-45b5-4677-8c56-f65f2d0f06d4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"
Aug 05 17:54:47.181169 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.181136    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q55d9\" (UniqueName: \"kubernetes.io/projected/4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458-kube-api-access-q55d9\") pod \"payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7\" (UID: \"4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458\") " pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"
Aug 05 17:54:47.273411 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.273357    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"
Aug 05 17:54:47.319226 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.319141    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"
Aug 05 17:54:47.322004 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.321934    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t" event={"ID":"546fbd81-44a9-4013-9c07-c7b64f3f3590","Type":"ContainerStarted","Data":"fb3891c409d40bab9e0464caf96339218171cd3dad44564bc47fecd82c445f7f"}
Aug 05 17:54:47.429209 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.429166    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"]
Aug 05 17:54:47.430632 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:54:47.430595    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4574a25a_7f1f_4f4f_a3b0_f8ee14d9e458.slice/crio-de742d7dd3a67ffbc63aef47714e95661551fabe740e5bd65ab1b3ab981c7286 WatchSource:0}: Error finding container de742d7dd3a67ffbc63aef47714e95661551fabe740e5bd65ab1b3ab981c7286: Status 404 returned error can't find the container with id de742d7dd3a67ffbc63aef47714e95661551fabe740e5bd65ab1b3ab981c7286
Aug 05 17:54:47.479604 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:47.479553    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"]
Aug 05 17:54:47.482110 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:54:47.482088    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod020d9235_45b5_4677_8c56_f65f2d0f06d4.slice/crio-91676262883b346b2a5946e7db43a8fe65d5b792a0a14c1ab7795c300b2be17e WatchSource:0}: Error finding container 91676262883b346b2a5946e7db43a8fe65d5b792a0a14c1ab7795c300b2be17e: Status 404 returned error can't find the container with id 91676262883b346b2a5946e7db43a8fe65d5b792a0a14c1ab7795c300b2be17e
Aug 05 17:54:48.329988 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:48.329951    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t" event={"ID":"546fbd81-44a9-4013-9c07-c7b64f3f3590","Type":"ContainerStarted","Data":"00fbbd10608c8aea5918ddbf2fce52fc29c6963697846cf58705b3d1974cb637"}
Aug 05 17:54:48.330460 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:48.330220    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"
Aug 05 17:54:48.331575 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:48.331546    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7" event={"ID":"4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458","Type":"ContainerStarted","Data":"00e3062034f302df331913945037028894b9ef9044ee8061bcb1381f2c932707"}
Aug 05 17:54:48.331707 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:48.331598    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"
Aug 05 17:54:48.331707 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:48.331621    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7" event={"ID":"4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458","Type":"ContainerStarted","Data":"de742d7dd3a67ffbc63aef47714e95661551fabe740e5bd65ab1b3ab981c7286"}
Aug 05 17:54:48.333044 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:48.333013    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk" event={"ID":"020d9235-45b5-4677-8c56-f65f2d0f06d4","Type":"ContainerStarted","Data":"687d885fecfa91fc46422e6ab4cd4d2c3e4d2f100e9988f2dfaae81fe5b84c6c"}
Aug 05 17:54:48.333201 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:48.333066    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk" event={"ID":"020d9235-45b5-4677-8c56-f65f2d0f06d4","Type":"ContainerStarted","Data":"91676262883b346b2a5946e7db43a8fe65d5b792a0a14c1ab7795c300b2be17e"}
Aug 05 17:54:48.333201 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:48.333136    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"
Aug 05 17:54:48.346082 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:48.346038    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t" podStartSLOduration=1.9841239960000001 podStartE2EDuration="2.346025597s" totalImagesPullingTime="361.901601ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:54:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:54:48.345643026 +0000 UTC m=+2101.763957789" watchObservedRunningTime="2026-08-05 17:54:48.346025597 +0000 UTC m=+2101.764340391"
Aug 05 17:54:48.362183 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:48.362126    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7" podStartSLOduration=2.3621142219999998 podStartE2EDuration="2.362114222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:54:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:54:48.360561233 +0000 UTC m=+2101.778876016" watchObservedRunningTime="2026-08-05 17:54:48.362114222 +0000 UTC m=+2101.780429021"
Aug 05 17:54:48.378508 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:48.378469    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk" podStartSLOduration=2.378456624 podStartE2EDuration="2.378456624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:54:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:54:48.377759941 +0000 UTC m=+2101.796074717" watchObservedRunningTime="2026-08-05 17:54:48.378456624 +0000 UTC m=+2101.796771380"
Aug 05 17:54:54.343481 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:54.343450    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"
Aug 05 17:54:59.339806 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:59.339758    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk" podUID="020d9235-45b5-4677-8c56-f65f2d0f06d4" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.57:9004: connect: connection refused"
Aug 05 17:54:59.340304 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:54:59.340286    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"
Aug 05 17:55:07.320426 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:07.320381    2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk" podUID="020d9235-45b5-4677-8c56-f65f2d0f06d4" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.57:9004: connect: connection refused"
Aug 05 17:55:09.339505 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:09.339473    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"
Aug 05 17:55:27.863206 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:27.863129    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"]
Aug 05 17:55:27.863830 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:27.863465    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t" podUID="546fbd81-44a9-4013-9c07-c7b64f3f3590" containerName="maas-api" containerID="cri-o://00fbbd10608c8aea5918ddbf2fce52fc29c6963697846cf58705b3d1974cb637" gracePeriod=30
Aug 05 17:55:27.949525 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:27.949444    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"]
Aug 05 17:55:27.949737 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:27.949704    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7" podUID="4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458" containerName="payload-processing" containerID="cri-o://00e3062034f302df331913945037028894b9ef9044ee8061bcb1381f2c932707" gracePeriod=30
Aug 05 17:55:27.956105 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:27.956080    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"]
Aug 05 17:55:27.956323 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:27.956302    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk" podUID="020d9235-45b5-4677-8c56-f65f2d0f06d4" containerName="payload-pre-processing" containerID="cri-o://687d885fecfa91fc46422e6ab4cd4d2c3e4d2f100e9988f2dfaae81fe5b84c6c" gracePeriod=30
Aug 05 17:55:28.218058 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.218028    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"
Aug 05 17:55:28.247269 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.247242    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"
Aug 05 17:55:28.250534 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.250506    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"
Aug 05 17:55:28.413730 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.413654    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/020d9235-45b5-4677-8c56-f65f2d0f06d4-plugins-config-volume\") pod \"020d9235-45b5-4677-8c56-f65f2d0f06d4\" (UID: \"020d9235-45b5-4677-8c56-f65f2d0f06d4\") "
Aug 05 17:55:28.413730 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.413713    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/546fbd81-44a9-4013-9c07-c7b64f3f3590-maas-api-tls\") pod \"546fbd81-44a9-4013-9c07-c7b64f3f3590\" (UID: \"546fbd81-44a9-4013-9c07-c7b64f3f3590\") "
Aug 05 17:55:28.413956 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.413773    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q55d9\" (UniqueName: \"kubernetes.io/projected/4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458-kube-api-access-q55d9\") pod \"4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458\" (UID: \"4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458\") "
Aug 05 17:55:28.413956 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.413805    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458-plugins-config-volume\") pod \"4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458\" (UID: \"4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458\") "
Aug 05 17:55:28.413956 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.413834    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8nnv\" (UniqueName: \"kubernetes.io/projected/546fbd81-44a9-4013-9c07-c7b64f3f3590-kube-api-access-n8nnv\") pod \"546fbd81-44a9-4013-9c07-c7b64f3f3590\" (UID: \"546fbd81-44a9-4013-9c07-c7b64f3f3590\") "
Aug 05 17:55:28.413956 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.413938    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mjnj\" (UniqueName: \"kubernetes.io/projected/020d9235-45b5-4677-8c56-f65f2d0f06d4-kube-api-access-2mjnj\") pod \"020d9235-45b5-4677-8c56-f65f2d0f06d4\" (UID: \"020d9235-45b5-4677-8c56-f65f2d0f06d4\") "
Aug 05 17:55:28.414196 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.413970    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/546fbd81-44a9-4013-9c07-c7b64f3f3590-openshift-service-ca\") pod \"546fbd81-44a9-4013-9c07-c7b64f3f3590\" (UID: \"546fbd81-44a9-4013-9c07-c7b64f3f3590\") "
Aug 05 17:55:28.414263 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.414232    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458" (UID: "4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:55:28.414394 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.414372    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/020d9235-45b5-4677-8c56-f65f2d0f06d4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "020d9235-45b5-4677-8c56-f65f2d0f06d4" (UID: "020d9235-45b5-4677-8c56-f65f2d0f06d4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:55:28.414570 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.414545    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/546fbd81-44a9-4013-9c07-c7b64f3f3590-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "546fbd81-44a9-4013-9c07-c7b64f3f3590" (UID: "546fbd81-44a9-4013-9c07-c7b64f3f3590"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:55:28.416013 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.415985    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458-kube-api-access-q55d9" (OuterVolumeSpecName: "kube-api-access-q55d9") pod "4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458" (UID: "4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458"). InnerVolumeSpecName "kube-api-access-q55d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:55:28.416289 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.416263    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/546fbd81-44a9-4013-9c07-c7b64f3f3590-kube-api-access-n8nnv" (OuterVolumeSpecName: "kube-api-access-n8nnv") pod "546fbd81-44a9-4013-9c07-c7b64f3f3590" (UID: "546fbd81-44a9-4013-9c07-c7b64f3f3590"). InnerVolumeSpecName "kube-api-access-n8nnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:55:28.416457 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.416435    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/020d9235-45b5-4677-8c56-f65f2d0f06d4-kube-api-access-2mjnj" (OuterVolumeSpecName: "kube-api-access-2mjnj") pod "020d9235-45b5-4677-8c56-f65f2d0f06d4" (UID: "020d9235-45b5-4677-8c56-f65f2d0f06d4"). InnerVolumeSpecName "kube-api-access-2mjnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:55:28.416520 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.416460    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/546fbd81-44a9-4013-9c07-c7b64f3f3590-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "546fbd81-44a9-4013-9c07-c7b64f3f3590" (UID: "546fbd81-44a9-4013-9c07-c7b64f3f3590"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:55:28.515140 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.515103    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/020d9235-45b5-4677-8c56-f65f2d0f06d4-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:55:28.515140 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.515128    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/546fbd81-44a9-4013-9c07-c7b64f3f3590-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:55:28.515140 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.515138    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q55d9\" (UniqueName: \"kubernetes.io/projected/4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458-kube-api-access-q55d9\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:55:28.515140 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.515159    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:55:28.515140 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.515168    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8nnv\" (UniqueName: \"kubernetes.io/projected/546fbd81-44a9-4013-9c07-c7b64f3f3590-kube-api-access-n8nnv\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:55:28.515140 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.515177    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mjnj\" (UniqueName: \"kubernetes.io/projected/020d9235-45b5-4677-8c56-f65f2d0f06d4-kube-api-access-2mjnj\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:55:28.515593 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.515185    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/546fbd81-44a9-4013-9c07-c7b64f3f3590-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:55:28.521292 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.521267    2556 generic.go:356] "Generic (PLEG): container finished" podID="546fbd81-44a9-4013-9c07-c7b64f3f3590" containerID="00fbbd10608c8aea5918ddbf2fce52fc29c6963697846cf58705b3d1974cb637" exitCode=0
Aug 05 17:55:28.521404 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.521309    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"
Aug 05 17:55:28.521404 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.521352    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t" event={"ID":"546fbd81-44a9-4013-9c07-c7b64f3f3590","Type":"ContainerDied","Data":"00fbbd10608c8aea5918ddbf2fce52fc29c6963697846cf58705b3d1974cb637"}
Aug 05 17:55:28.521523 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.521435    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t" event={"ID":"546fbd81-44a9-4013-9c07-c7b64f3f3590","Type":"ContainerDied","Data":"fb3891c409d40bab9e0464caf96339218171cd3dad44564bc47fecd82c445f7f"}
Aug 05 17:55:28.521523 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.521469    2556 scope.go:117] "RemoveContainer" containerID="00fbbd10608c8aea5918ddbf2fce52fc29c6963697846cf58705b3d1974cb637"
Aug 05 17:55:28.522753 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.522728    2556 generic.go:356] "Generic (PLEG): container finished" podID="4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458" containerID="00e3062034f302df331913945037028894b9ef9044ee8061bcb1381f2c932707" exitCode=0
Aug 05 17:55:28.522864 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.522776    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"
Aug 05 17:55:28.522864 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.522822    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7" event={"ID":"4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458","Type":"ContainerDied","Data":"00e3062034f302df331913945037028894b9ef9044ee8061bcb1381f2c932707"}
Aug 05 17:55:28.522981 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.522873    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7" event={"ID":"4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458","Type":"ContainerDied","Data":"de742d7dd3a67ffbc63aef47714e95661551fabe740e5bd65ab1b3ab981c7286"}
Aug 05 17:55:28.524222 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.524199    2556 generic.go:356] "Generic (PLEG): container finished" podID="020d9235-45b5-4677-8c56-f65f2d0f06d4" containerID="687d885fecfa91fc46422e6ab4cd4d2c3e4d2f100e9988f2dfaae81fe5b84c6c" exitCode=0
Aug 05 17:55:28.524336 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.524236    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"
Aug 05 17:55:28.524336 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.524264    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk" event={"ID":"020d9235-45b5-4677-8c56-f65f2d0f06d4","Type":"ContainerDied","Data":"687d885fecfa91fc46422e6ab4cd4d2c3e4d2f100e9988f2dfaae81fe5b84c6c"}
Aug 05 17:55:28.524336 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.524299    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk" event={"ID":"020d9235-45b5-4677-8c56-f65f2d0f06d4","Type":"ContainerDied","Data":"91676262883b346b2a5946e7db43a8fe65d5b792a0a14c1ab7795c300b2be17e"}
Aug 05 17:55:28.532301 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.532067    2556 scope.go:117] "RemoveContainer" containerID="00fbbd10608c8aea5918ddbf2fce52fc29c6963697846cf58705b3d1974cb637"
Aug 05 17:55:28.532477 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:55:28.532451    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00fbbd10608c8aea5918ddbf2fce52fc29c6963697846cf58705b3d1974cb637\": container with ID starting with 00fbbd10608c8aea5918ddbf2fce52fc29c6963697846cf58705b3d1974cb637 not found: ID does not exist" containerID="00fbbd10608c8aea5918ddbf2fce52fc29c6963697846cf58705b3d1974cb637"
Aug 05 17:55:28.532570 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.532485    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00fbbd10608c8aea5918ddbf2fce52fc29c6963697846cf58705b3d1974cb637"} err="failed to get container status \"00fbbd10608c8aea5918ddbf2fce52fc29c6963697846cf58705b3d1974cb637\": rpc error: code = NotFound desc = could not find container \"00fbbd10608c8aea5918ddbf2fce52fc29c6963697846cf58705b3d1974cb637\": container with ID starting with 00fbbd10608c8aea5918ddbf2fce52fc29c6963697846cf58705b3d1974cb637 not found: ID does not exist"
Aug 05 17:55:28.532570 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.532508    2556 scope.go:117] "RemoveContainer" containerID="00e3062034f302df331913945037028894b9ef9044ee8061bcb1381f2c932707"
Aug 05 17:55:28.540319 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.540304    2556 scope.go:117] "RemoveContainer" containerID="00e3062034f302df331913945037028894b9ef9044ee8061bcb1381f2c932707"
Aug 05 17:55:28.540551 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:55:28.540534    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00e3062034f302df331913945037028894b9ef9044ee8061bcb1381f2c932707\": container with ID starting with 00e3062034f302df331913945037028894b9ef9044ee8061bcb1381f2c932707 not found: ID does not exist" containerID="00e3062034f302df331913945037028894b9ef9044ee8061bcb1381f2c932707"
Aug 05 17:55:28.540621 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.540561    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00e3062034f302df331913945037028894b9ef9044ee8061bcb1381f2c932707"} err="failed to get container status \"00e3062034f302df331913945037028894b9ef9044ee8061bcb1381f2c932707\": rpc error: code = NotFound desc = could not find container \"00e3062034f302df331913945037028894b9ef9044ee8061bcb1381f2c932707\": container with ID starting with 00e3062034f302df331913945037028894b9ef9044ee8061bcb1381f2c932707 not found: ID does not exist"
Aug 05 17:55:28.540621 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.540583    2556 scope.go:117] "RemoveContainer" containerID="687d885fecfa91fc46422e6ab4cd4d2c3e4d2f100e9988f2dfaae81fe5b84c6c"
Aug 05 17:55:28.545820 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.545800    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"]
Aug 05 17:55:28.548716 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.548688    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f48a67f-6986857d95-qvf4t"]
Aug 05 17:55:28.548996 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.548978    2556 scope.go:117] "RemoveContainer" containerID="687d885fecfa91fc46422e6ab4cd4d2c3e4d2f100e9988f2dfaae81fe5b84c6c"
Aug 05 17:55:28.549288 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:55:28.549267    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"687d885fecfa91fc46422e6ab4cd4d2c3e4d2f100e9988f2dfaae81fe5b84c6c\": container with ID starting with 687d885fecfa91fc46422e6ab4cd4d2c3e4d2f100e9988f2dfaae81fe5b84c6c not found: ID does not exist" containerID="687d885fecfa91fc46422e6ab4cd4d2c3e4d2f100e9988f2dfaae81fe5b84c6c"
Aug 05 17:55:28.549372 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.549293    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"687d885fecfa91fc46422e6ab4cd4d2c3e4d2f100e9988f2dfaae81fe5b84c6c"} err="failed to get container status \"687d885fecfa91fc46422e6ab4cd4d2c3e4d2f100e9988f2dfaae81fe5b84c6c\": rpc error: code = NotFound desc = could not find container \"687d885fecfa91fc46422e6ab4cd4d2c3e4d2f100e9988f2dfaae81fe5b84c6c\": container with ID starting with 687d885fecfa91fc46422e6ab4cd4d2c3e4d2f100e9988f2dfaae81fe5b84c6c not found: ID does not exist"
Aug 05 17:55:28.557328 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.557309    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"]
Aug 05 17:55:28.562745 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.562723    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4f48a67f-54ff498b4-r6qhk"]
Aug 05 17:55:28.574965 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.574945    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"]
Aug 05 17:55:28.580540 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:28.580522    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4f48a67f-f868c5f86-dsdm7"]
Aug 05 17:55:29.097355 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:29.097324    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="020d9235-45b5-4677-8c56-f65f2d0f06d4" path="/var/lib/kubelet/pods/020d9235-45b5-4677-8c56-f65f2d0f06d4/volumes"
Aug 05 17:55:29.097689 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:29.097672    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458" path="/var/lib/kubelet/pods/4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458/volumes"
Aug 05 17:55:29.098022 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:55:29.097998    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="546fbd81-44a9-4013-9c07-c7b64f3f3590" path="/var/lib/kubelet/pods/546fbd81-44a9-4013-9c07-c7b64f3f3590/volumes"
Aug 05 17:56:15.296670 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.296632    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"]
Aug 05 17:56:15.297653 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.296847    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w" podUID="3b4ddea7-f22a-4077-b9c8-c47857ac3876" containerName="manager" containerID="cri-o://cbc757989a293d5c54fffdbb2fe401d84de031286a4662688bbbdfe6c103977d" gracePeriod=10
Aug 05 17:56:15.549975 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.549921    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"
Aug 05 17:56:15.685090 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.685059    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b4ddea7-f22a-4077-b9c8-c47857ac3876-extensions-socket-volume\") pod \"3b4ddea7-f22a-4077-b9c8-c47857ac3876\" (UID: \"3b4ddea7-f22a-4077-b9c8-c47857ac3876\") "
Aug 05 17:56:15.685294 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.685175    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhb8m\" (UniqueName: \"kubernetes.io/projected/3b4ddea7-f22a-4077-b9c8-c47857ac3876-kube-api-access-vhb8m\") pod \"3b4ddea7-f22a-4077-b9c8-c47857ac3876\" (UID: \"3b4ddea7-f22a-4077-b9c8-c47857ac3876\") "
Aug 05 17:56:15.685501 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.685475    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b4ddea7-f22a-4077-b9c8-c47857ac3876-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3b4ddea7-f22a-4077-b9c8-c47857ac3876" (UID: "3b4ddea7-f22a-4077-b9c8-c47857ac3876"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:56:15.687107 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.687079    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b4ddea7-f22a-4077-b9c8-c47857ac3876-kube-api-access-vhb8m" (OuterVolumeSpecName: "kube-api-access-vhb8m") pod "3b4ddea7-f22a-4077-b9c8-c47857ac3876" (UID: "3b4ddea7-f22a-4077-b9c8-c47857ac3876"). InnerVolumeSpecName "kube-api-access-vhb8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:56:15.741976 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.741948    2556 generic.go:356] "Generic (PLEG): container finished" podID="3b4ddea7-f22a-4077-b9c8-c47857ac3876" containerID="cbc757989a293d5c54fffdbb2fe401d84de031286a4662688bbbdfe6c103977d" exitCode=0
Aug 05 17:56:15.742087 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.741982    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w" event={"ID":"3b4ddea7-f22a-4077-b9c8-c47857ac3876","Type":"ContainerDied","Data":"cbc757989a293d5c54fffdbb2fe401d84de031286a4662688bbbdfe6c103977d"}
Aug 05 17:56:15.742087 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.741989    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"
Aug 05 17:56:15.742087 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.742020    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w" event={"ID":"3b4ddea7-f22a-4077-b9c8-c47857ac3876","Type":"ContainerDied","Data":"a51a2bc494c848dbb766b1f70f873ca47171b77d4c91e5c1a539a99ede01af9d"}
Aug 05 17:56:15.742087 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.742040    2556 scope.go:117] "RemoveContainer" containerID="cbc757989a293d5c54fffdbb2fe401d84de031286a4662688bbbdfe6c103977d"
Aug 05 17:56:15.752038 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.752021    2556 scope.go:117] "RemoveContainer" containerID="cbc757989a293d5c54fffdbb2fe401d84de031286a4662688bbbdfe6c103977d"
Aug 05 17:56:15.752349 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:56:15.752330    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbc757989a293d5c54fffdbb2fe401d84de031286a4662688bbbdfe6c103977d\": container with ID starting with cbc757989a293d5c54fffdbb2fe401d84de031286a4662688bbbdfe6c103977d not found: ID does not exist" containerID="cbc757989a293d5c54fffdbb2fe401d84de031286a4662688bbbdfe6c103977d"
Aug 05 17:56:15.752419 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.752356    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbc757989a293d5c54fffdbb2fe401d84de031286a4662688bbbdfe6c103977d"} err="failed to get container status \"cbc757989a293d5c54fffdbb2fe401d84de031286a4662688bbbdfe6c103977d\": rpc error: code = NotFound desc = could not find container \"cbc757989a293d5c54fffdbb2fe401d84de031286a4662688bbbdfe6c103977d\": container with ID starting with cbc757989a293d5c54fffdbb2fe401d84de031286a4662688bbbdfe6c103977d not found: ID does not exist"
Aug 05 17:56:15.763892 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.763872    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"]
Aug 05 17:56:15.767281 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.767264    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-fpj7w"]
Aug 05 17:56:15.785963 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.785940    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhb8m\" (UniqueName: \"kubernetes.io/projected/3b4ddea7-f22a-4077-b9c8-c47857ac3876-kube-api-access-vhb8m\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:56:15.786045 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:15.785963    2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b4ddea7-f22a-4077-b9c8-c47857ac3876-extensions-socket-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:56:17.097161 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:56:17.097128    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b4ddea7-f22a-4077-b9c8-c47857ac3876" path="/var/lib/kubelet/pods/3b4ddea7-f22a-4077-b9c8-c47857ac3876/volumes"
Aug 05 17:57:30.943468 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.943390    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2"]
Aug 05 17:57:30.943942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.943727    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="020d9235-45b5-4677-8c56-f65f2d0f06d4" containerName="payload-pre-processing"
Aug 05 17:57:30.943942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.943739    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="020d9235-45b5-4677-8c56-f65f2d0f06d4" containerName="payload-pre-processing"
Aug 05 17:57:30.943942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.943753    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458" containerName="payload-processing"
Aug 05 17:57:30.943942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.943759    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458" containerName="payload-processing"
Aug 05 17:57:30.943942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.943773    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="546fbd81-44a9-4013-9c07-c7b64f3f3590" containerName="maas-api"
Aug 05 17:57:30.943942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.943778    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="546fbd81-44a9-4013-9c07-c7b64f3f3590" containerName="maas-api"
Aug 05 17:57:30.943942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.943786    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b4ddea7-f22a-4077-b9c8-c47857ac3876" containerName="manager"
Aug 05 17:57:30.943942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.943791    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b4ddea7-f22a-4077-b9c8-c47857ac3876" containerName="manager"
Aug 05 17:57:30.943942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.943843    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b4ddea7-f22a-4077-b9c8-c47857ac3876" containerName="manager"
Aug 05 17:57:30.943942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.943851    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4574a25a-7f1f-4f4f-a3b0-f8ee14d9e458" containerName="payload-processing"
Aug 05 17:57:30.943942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.943858    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="020d9235-45b5-4677-8c56-f65f2d0f06d4" containerName="payload-pre-processing"
Aug 05 17:57:30.943942 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.943863    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="546fbd81-44a9-4013-9c07-c7b64f3f3590" containerName="maas-api"
Aug 05 17:57:30.948069 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.948041    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2"
Aug 05 17:57:30.951079 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.951049    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nh2zq\""
Aug 05 17:57:30.958843 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:30.958824    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2"]
Aug 05 17:57:31.053213 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:31.053182    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73cf224c-963a-4dd4-bc75-102512457040-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-8v9h2\" (UID: \"73cf224c-963a-4dd4-bc75-102512457040\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2"
Aug 05 17:57:31.053361 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:31.053241    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksvjs\" (UniqueName: \"kubernetes.io/projected/73cf224c-963a-4dd4-bc75-102512457040-kube-api-access-ksvjs\") pod \"kuadrant-operator-controller-manager-68cf589589-8v9h2\" (UID: \"73cf224c-963a-4dd4-bc75-102512457040\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2"
Aug 05 17:57:31.154516 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:31.154487    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73cf224c-963a-4dd4-bc75-102512457040-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-8v9h2\" (UID: \"73cf224c-963a-4dd4-bc75-102512457040\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2"
Aug 05 17:57:31.154660 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:31.154537    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksvjs\" (UniqueName: \"kubernetes.io/projected/73cf224c-963a-4dd4-bc75-102512457040-kube-api-access-ksvjs\") pod \"kuadrant-operator-controller-manager-68cf589589-8v9h2\" (UID: \"73cf224c-963a-4dd4-bc75-102512457040\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2"
Aug 05 17:57:31.154922 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:31.154904    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73cf224c-963a-4dd4-bc75-102512457040-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-8v9h2\" (UID: \"73cf224c-963a-4dd4-bc75-102512457040\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2"
Aug 05 17:57:31.162764 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:31.162741    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksvjs\" (UniqueName: \"kubernetes.io/projected/73cf224c-963a-4dd4-bc75-102512457040-kube-api-access-ksvjs\") pod \"kuadrant-operator-controller-manager-68cf589589-8v9h2\" (UID: \"73cf224c-963a-4dd4-bc75-102512457040\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2"
Aug 05 17:57:31.261901 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:31.261869    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2"
Aug 05 17:57:31.398541 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:31.398515    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2"]
Aug 05 17:57:31.400641 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:57:31.400617    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73cf224c_963a_4dd4_bc75_102512457040.slice/crio-bd5f08356e93570de08dc6d1c3d43e795c332114e61781f2c0769bca44ad3d88 WatchSource:0}: Error finding container bd5f08356e93570de08dc6d1c3d43e795c332114e61781f2c0769bca44ad3d88: Status 404 returned error can't find the container with id bd5f08356e93570de08dc6d1c3d43e795c332114e61781f2c0769bca44ad3d88
Aug 05 17:57:32.078944 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:32.078907    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2" event={"ID":"73cf224c-963a-4dd4-bc75-102512457040","Type":"ContainerStarted","Data":"5d038d8939ec428cc3ad1101f16c59ea1b4587928bbee40b5fdd3cc2c71f9e30"}
Aug 05 17:57:32.079324 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:32.078965    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2" event={"ID":"73cf224c-963a-4dd4-bc75-102512457040","Type":"ContainerStarted","Data":"bd5f08356e93570de08dc6d1c3d43e795c332114e61781f2c0769bca44ad3d88"}
Aug 05 17:57:32.079324 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:32.078993    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2"
Aug 05 17:57:32.120341 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:32.120287    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2" podStartSLOduration=2.120267831 podStartE2EDuration="2.120267831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:57:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:57:32.118279174 +0000 UTC m=+2265.536593964" watchObservedRunningTime="2026-08-05 17:57:32.120267831 +0000 UTC m=+2265.538582591"
Aug 05 17:57:43.086013 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:43.085985    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8v9h2"
Aug 05 17:57:54.071492 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.071453    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"]
Aug 05 17:57:54.075980 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.075960    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"
Aug 05 17:57:54.078435 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.078413    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b60f38ad-serving-cert\""
Aug 05 17:57:54.078710 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.078512    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ql6kj\""
Aug 05 17:57:54.082813 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.082787    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"]
Aug 05 17:57:54.239639 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.239605    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djrr6\" (UniqueName: \"kubernetes.io/projected/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-kube-api-access-djrr6\") pod \"maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h\" (UID: \"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"
Aug 05 17:57:54.239815 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.239648    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-maas-api-tls\") pod \"maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h\" (UID: \"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"
Aug 05 17:57:54.239815 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.239681    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-openshift-service-ca\") pod \"maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h\" (UID: \"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"
Aug 05 17:57:54.340815 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.340727    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djrr6\" (UniqueName: \"kubernetes.io/projected/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-kube-api-access-djrr6\") pod \"maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h\" (UID: \"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"
Aug 05 17:57:54.340815 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.340772    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-maas-api-tls\") pod \"maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h\" (UID: \"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"
Aug 05 17:57:54.340815 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.340803    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-openshift-service-ca\") pod \"maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h\" (UID: \"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"
Aug 05 17:57:54.341558 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.341537    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-openshift-service-ca\") pod \"maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h\" (UID: \"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"
Aug 05 17:57:54.343396 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.343375    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-maas-api-tls\") pod \"maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h\" (UID: \"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"
Aug 05 17:57:54.348399 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.348371    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djrr6\" (UniqueName: \"kubernetes.io/projected/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-kube-api-access-djrr6\") pod \"maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h\" (UID: \"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"
Aug 05 17:57:54.389343 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.389318    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"
Aug 05 17:57:54.528508 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.528472    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"]
Aug 05 17:57:54.532177 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:57:54.532128    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c1cfa2e_4b89_4cd9_85ff_b575aa47bd64.slice/crio-4cc6982a7c9e89fec5d673a9b9f65504c8153902c068ac4940e4b5f57ce4e7da WatchSource:0}: Error finding container 4cc6982a7c9e89fec5d673a9b9f65504c8153902c068ac4940e4b5f57ce4e7da: Status 404 returned error can't find the container with id 4cc6982a7c9e89fec5d673a9b9f65504c8153902c068ac4940e4b5f57ce4e7da
Aug 05 17:57:54.533730 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.533709    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:57:54.771975 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.771941    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"]
Aug 05 17:57:54.776328 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.776311    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"
Aug 05 17:57:54.778960 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.778940    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-b60f38ad-dockercfg-4h9k4\""
Aug 05 17:57:54.779083 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.778978    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-b60f38ad\""
Aug 05 17:57:54.785884 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.784926    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"]
Aug 05 17:57:54.816106 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.816078    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"]
Aug 05 17:57:54.819351 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.819335    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"
Aug 05 17:57:54.830174 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.830131    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"]
Aug 05 17:57:54.845788 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.845758    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ece7846d-612a-4f8b-99ba-c532c4f55010-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9\" (UID: \"ece7846d-612a-4f8b-99ba-c532c4f55010\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"
Aug 05 17:57:54.845880 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.845819    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4hfc\" (UniqueName: \"kubernetes.io/projected/ece7846d-612a-4f8b-99ba-c532c4f55010-kube-api-access-c4hfc\") pod \"payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9\" (UID: \"ece7846d-612a-4f8b-99ba-c532c4f55010\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"
Aug 05 17:57:54.947305 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.947276    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ece7846d-612a-4f8b-99ba-c532c4f55010-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9\" (UID: \"ece7846d-612a-4f8b-99ba-c532c4f55010\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"
Aug 05 17:57:54.947425 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.947330    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxnf7\" (UniqueName: \"kubernetes.io/projected/feab9c5d-7711-494f-9676-42a1aa8b183d-kube-api-access-fxnf7\") pod \"payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl\" (UID: \"feab9c5d-7711-494f-9676-42a1aa8b183d\") " pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"
Aug 05 17:57:54.947425 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.947369    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4hfc\" (UniqueName: \"kubernetes.io/projected/ece7846d-612a-4f8b-99ba-c532c4f55010-kube-api-access-c4hfc\") pod \"payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9\" (UID: \"ece7846d-612a-4f8b-99ba-c532c4f55010\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"
Aug 05 17:57:54.947425 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.947404    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feab9c5d-7711-494f-9676-42a1aa8b183d-plugins-config-volume\") pod \"payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl\" (UID: \"feab9c5d-7711-494f-9676-42a1aa8b183d\") " pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"
Aug 05 17:57:54.948290 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.948263    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ece7846d-612a-4f8b-99ba-c532c4f55010-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9\" (UID: \"ece7846d-612a-4f8b-99ba-c532c4f55010\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"
Aug 05 17:57:54.954505 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:54.954482    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4hfc\" (UniqueName: \"kubernetes.io/projected/ece7846d-612a-4f8b-99ba-c532c4f55010-kube-api-access-c4hfc\") pod \"payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9\" (UID: \"ece7846d-612a-4f8b-99ba-c532c4f55010\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"
Aug 05 17:57:55.048207 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:55.048117    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxnf7\" (UniqueName: \"kubernetes.io/projected/feab9c5d-7711-494f-9676-42a1aa8b183d-kube-api-access-fxnf7\") pod \"payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl\" (UID: \"feab9c5d-7711-494f-9676-42a1aa8b183d\") " pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"
Aug 05 17:57:55.048207 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:55.048181    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feab9c5d-7711-494f-9676-42a1aa8b183d-plugins-config-volume\") pod \"payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl\" (UID: \"feab9c5d-7711-494f-9676-42a1aa8b183d\") " pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"
Aug 05 17:57:55.048840 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:55.048820    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feab9c5d-7711-494f-9676-42a1aa8b183d-plugins-config-volume\") pod \"payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl\" (UID: \"feab9c5d-7711-494f-9676-42a1aa8b183d\") " pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"
Aug 05 17:57:55.055582 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:55.055556    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxnf7\" (UniqueName: \"kubernetes.io/projected/feab9c5d-7711-494f-9676-42a1aa8b183d-kube-api-access-fxnf7\") pod \"payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl\" (UID: \"feab9c5d-7711-494f-9676-42a1aa8b183d\") " pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"
Aug 05 17:57:55.094492 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:55.094471    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"
Aug 05 17:57:55.130490 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:55.130458    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"
Aug 05 17:57:55.188986 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:55.188948    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h" event={"ID":"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64","Type":"ContainerStarted","Data":"c3d3aaf2aa08143cd35992c14c8f2c5ad13df9bb10792b335afbfdace9077e2e"}
Aug 05 17:57:55.189097 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:55.189019    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h" event={"ID":"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64","Type":"ContainerStarted","Data":"4cc6982a7c9e89fec5d673a9b9f65504c8153902c068ac4940e4b5f57ce4e7da"}
Aug 05 17:57:55.189411 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:55.189315    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"
Aug 05 17:57:55.207205 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:55.207025    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h" podStartSLOduration=0.851513148 podStartE2EDuration="1.207008502s" totalImagesPullingTime="355.495354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:57:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:57:55.205040766 +0000 UTC m=+2288.623355524" watchObservedRunningTime="2026-08-05 17:57:55.207008502 +0000 UTC m=+2288.625323259"
Aug 05 17:57:55.271873 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:55.271846    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"]
Aug 05 17:57:55.274121 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:57:55.274078    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece7846d_612a_4f8b_99ba_c532c4f55010.slice/crio-fa07b4c60916e61b925b07c8aa03cca4b5ccf72368f69f88c2a59dd77e02c472 WatchSource:0}: Error finding container fa07b4c60916e61b925b07c8aa03cca4b5ccf72368f69f88c2a59dd77e02c472: Status 404 returned error can't find the container with id fa07b4c60916e61b925b07c8aa03cca4b5ccf72368f69f88c2a59dd77e02c472
Aug 05 17:57:55.449684 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:55.449660    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"]
Aug 05 17:57:55.451844 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:57:55.451808    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeab9c5d_7711_494f_9676_42a1aa8b183d.slice/crio-4460b5d67a2dcc7abf930ab2a3add85c8a34c4195b70666c60ff4c5f55a2e1f4 WatchSource:0}: Error finding container 4460b5d67a2dcc7abf930ab2a3add85c8a34c4195b70666c60ff4c5f55a2e1f4: Status 404 returned error can't find the container with id 4460b5d67a2dcc7abf930ab2a3add85c8a34c4195b70666c60ff4c5f55a2e1f4
Aug 05 17:57:56.195224 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:56.195186    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl" event={"ID":"feab9c5d-7711-494f-9676-42a1aa8b183d","Type":"ContainerStarted","Data":"e02e2500a08708412a12916f15ba6607938b1ce51f2974361c38ed091a98dbe4"}
Aug 05 17:57:56.195681 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:56.195246    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl" event={"ID":"feab9c5d-7711-494f-9676-42a1aa8b183d","Type":"ContainerStarted","Data":"4460b5d67a2dcc7abf930ab2a3add85c8a34c4195b70666c60ff4c5f55a2e1f4"}
Aug 05 17:57:56.195681 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:56.195288    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"
Aug 05 17:57:56.196923 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:56.196895    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9" event={"ID":"ece7846d-612a-4f8b-99ba-c532c4f55010","Type":"ContainerStarted","Data":"1d5cf1310f70bf62e80f329b3804bb1e4c7266fa2b57835c17f55f8c8826e198"}
Aug 05 17:57:56.197019 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:56.196942    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9" event={"ID":"ece7846d-612a-4f8b-99ba-c532c4f55010","Type":"ContainerStarted","Data":"fa07b4c60916e61b925b07c8aa03cca4b5ccf72368f69f88c2a59dd77e02c472"}
Aug 05 17:57:56.213540 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:56.213488    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl" podStartSLOduration=2.213468449 podStartE2EDuration="2.213468449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:57:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:57:56.210182276 +0000 UTC m=+2289.628497064" watchObservedRunningTime="2026-08-05 17:57:56.213468449 +0000 UTC m=+2289.631783207"
Aug 05 17:57:56.226573 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:56.226525    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9" podStartSLOduration=2.226507082 podStartE2EDuration="2.226507082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:57:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:57:56.224060128 +0000 UTC m=+2289.642374907" watchObservedRunningTime="2026-08-05 17:57:56.226507082 +0000 UTC m=+2289.644821839"
Aug 05 17:57:57.201871 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:57:57.201831    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"
Aug 05 17:58:01.202606 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:01.202578    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"
Aug 05 17:58:07.203994 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:07.203967    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"
Aug 05 17:58:08.207975 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:08.207947    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"
Aug 05 17:58:37.796840 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.796807    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"]
Aug 05 17:58:37.800859 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.800839    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.804136 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.804110    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b60f38ad\"/\"e2e-lifecycle-model-b60f38ad-kserve-self-signed-certs\""
Aug 05 17:58:37.804136 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.804120    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b60f38ad\"/\"openshift-service-ca.crt\""
Aug 05 17:58:37.804315 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.804142    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b60f38ad\"/\"default-dockercfg-xzg4d\""
Aug 05 17:58:37.804315 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.804198    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b60f38ad\"/\"kube-root-ca.crt\""
Aug 05 17:58:37.808365 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.808344    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"]
Aug 05 17:58:37.876011 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.875978    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-model-cache\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.876011 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.876013    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-tmp-dir\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.876235 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.876032    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32d3d762-856b-4942-9b2c-ea513157de8e-tls-certs\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.876235 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.876047    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlgtc\" (UniqueName: \"kubernetes.io/projected/32d3d762-856b-4942-9b2c-ea513157de8e-kube-api-access-zlgtc\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.876235 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.876175    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-home\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.876235 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.876226    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-dshm\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.977123 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.977091    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-model-cache\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.977123 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.977124    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-tmp-dir\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.977353 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.977143    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32d3d762-856b-4942-9b2c-ea513157de8e-tls-certs\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.977353 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.977193    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlgtc\" (UniqueName: \"kubernetes.io/projected/32d3d762-856b-4942-9b2c-ea513157de8e-kube-api-access-zlgtc\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.977353 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.977236    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-home\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.977353 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.977272    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-dshm\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.977583 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.977562    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-model-cache\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.977643 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.977600    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-tmp-dir\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.977643 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.977626    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-home\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.979484 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.979466    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-dshm\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.979762 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.979743    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32d3d762-856b-4942-9b2c-ea513157de8e-tls-certs\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:37.985530 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:37.985508    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlgtc\" (UniqueName: \"kubernetes.io/projected/32d3d762-856b-4942-9b2c-ea513157de8e-kube-api-access-zlgtc\") pod \"e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") " pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:38.110661 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:38.110603    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:38.241522 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:38.241496    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"]
Aug 05 17:58:38.242983 ip-10-0-141-37 kubenswrapper[2556]: W0805 17:58:38.242947    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32d3d762_856b_4942_9b2c_ea513157de8e.slice/crio-061d11ee62795631e1b279876eb41b7a774d1f0860bf4b9eb902fe09959b11e0 WatchSource:0}: Error finding container 061d11ee62795631e1b279876eb41b7a774d1f0860bf4b9eb902fe09959b11e0: Status 404 returned error can't find the container with id 061d11ee62795631e1b279876eb41b7a774d1f0860bf4b9eb902fe09959b11e0
Aug 05 17:58:38.391477 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:38.391400    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc" event={"ID":"32d3d762-856b-4942-9b2c-ea513157de8e","Type":"ContainerStarted","Data":"48e93b888c6f4f967d25982d0e904077aab3c68d56881a58a5b3152ff9e44d1d"}
Aug 05 17:58:38.391477 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:38.391448    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc" event={"ID":"32d3d762-856b-4942-9b2c-ea513157de8e","Type":"ContainerStarted","Data":"061d11ee62795631e1b279876eb41b7a774d1f0860bf4b9eb902fe09959b11e0"}
Aug 05 17:58:38.407974 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:38.407722    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc" podStartSLOduration=1.407702929 podStartE2EDuration="1.407702929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:58:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:58:38.407508276 +0000 UTC m=+2331.825823036" watchObservedRunningTime="2026-08-05 17:58:38.407702929 +0000 UTC m=+2331.826017688"
Aug 05 17:58:48.111169 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:48.111072    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:48.111169 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:48.111125    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:48.123611 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:48.123586    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:58:48.452675 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:58:48.452592    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:59:18.539375 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:18.539344    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"]
Aug 05 17:59:18.539994 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:18.539533    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h" podUID="9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64" containerName="maas-api" containerID="cri-o://c3d3aaf2aa08143cd35992c14c8f2c5ad13df9bb10792b335afbfdace9077e2e" gracePeriod=30
Aug 05 17:59:18.605479 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:18.605438    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"]
Aug 05 17:59:18.605772 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:18.605702    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl" podUID="feab9c5d-7711-494f-9676-42a1aa8b183d" containerName="payload-processing" containerID="cri-o://e02e2500a08708412a12916f15ba6607938b1ce51f2974361c38ed091a98dbe4" gracePeriod=30
Aug 05 17:59:18.610858 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:18.610813    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"]
Aug 05 17:59:18.611326 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:18.611266    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9" podUID="ece7846d-612a-4f8b-99ba-c532c4f55010" containerName="payload-pre-processing" containerID="cri-o://1d5cf1310f70bf62e80f329b3804bb1e4c7266fa2b57835c17f55f8c8826e198" gracePeriod=30
Aug 05 17:59:18.894952 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:18.894921    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"
Aug 05 17:59:18.905284 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:18.905258    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"
Aug 05 17:59:18.918102 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:18.918037    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"
Aug 05 17:59:19.002474 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.002441    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feab9c5d-7711-494f-9676-42a1aa8b183d-plugins-config-volume\") pod \"feab9c5d-7711-494f-9676-42a1aa8b183d\" (UID: \"feab9c5d-7711-494f-9676-42a1aa8b183d\") "
Aug 05 17:59:19.002650 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.002499    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djrr6\" (UniqueName: \"kubernetes.io/projected/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-kube-api-access-djrr6\") pod \"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64\" (UID: \"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64\") "
Aug 05 17:59:19.002650 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.002517    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxnf7\" (UniqueName: \"kubernetes.io/projected/feab9c5d-7711-494f-9676-42a1aa8b183d-kube-api-access-fxnf7\") pod \"feab9c5d-7711-494f-9676-42a1aa8b183d\" (UID: \"feab9c5d-7711-494f-9676-42a1aa8b183d\") "
Aug 05 17:59:19.002650 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.002554    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-maas-api-tls\") pod \"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64\" (UID: \"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64\") "
Aug 05 17:59:19.002650 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.002577    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ece7846d-612a-4f8b-99ba-c532c4f55010-plugins-config-volume\") pod \"ece7846d-612a-4f8b-99ba-c532c4f55010\" (UID: \"ece7846d-612a-4f8b-99ba-c532c4f55010\") "
Aug 05 17:59:19.002650 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.002601    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4hfc\" (UniqueName: \"kubernetes.io/projected/ece7846d-612a-4f8b-99ba-c532c4f55010-kube-api-access-c4hfc\") pod \"ece7846d-612a-4f8b-99ba-c532c4f55010\" (UID: \"ece7846d-612a-4f8b-99ba-c532c4f55010\") "
Aug 05 17:59:19.002650 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.002635    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-openshift-service-ca\") pod \"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64\" (UID: \"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64\") "
Aug 05 17:59:19.002981 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.002859    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feab9c5d-7711-494f-9676-42a1aa8b183d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "feab9c5d-7711-494f-9676-42a1aa8b183d" (UID: "feab9c5d-7711-494f-9676-42a1aa8b183d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:59:19.003169 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.003114    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ece7846d-612a-4f8b-99ba-c532c4f55010-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ece7846d-612a-4f8b-99ba-c532c4f55010" (UID: "ece7846d-612a-4f8b-99ba-c532c4f55010"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:59:19.003301 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.003134    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64" (UID: "9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:59:19.004853 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.004831    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-kube-api-access-djrr6" (OuterVolumeSpecName: "kube-api-access-djrr6") pod "9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64" (UID: "9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64"). InnerVolumeSpecName "kube-api-access-djrr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:59:19.004948 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.004883    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ece7846d-612a-4f8b-99ba-c532c4f55010-kube-api-access-c4hfc" (OuterVolumeSpecName: "kube-api-access-c4hfc") pod "ece7846d-612a-4f8b-99ba-c532c4f55010" (UID: "ece7846d-612a-4f8b-99ba-c532c4f55010"). InnerVolumeSpecName "kube-api-access-c4hfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:59:19.005114 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.005087    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feab9c5d-7711-494f-9676-42a1aa8b183d-kube-api-access-fxnf7" (OuterVolumeSpecName: "kube-api-access-fxnf7") pod "feab9c5d-7711-494f-9676-42a1aa8b183d" (UID: "feab9c5d-7711-494f-9676-42a1aa8b183d"). InnerVolumeSpecName "kube-api-access-fxnf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:59:19.005221 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.005111    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64" (UID: "9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:59:19.103274 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.103208    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:59:19.103274 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.103235    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ece7846d-612a-4f8b-99ba-c532c4f55010-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:59:19.103274 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.103248    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4hfc\" (UniqueName: \"kubernetes.io/projected/ece7846d-612a-4f8b-99ba-c532c4f55010-kube-api-access-c4hfc\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:59:19.103274 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.103261    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:59:19.103274 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.103274    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feab9c5d-7711-494f-9676-42a1aa8b183d-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:59:19.103521 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.103286    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djrr6\" (UniqueName: \"kubernetes.io/projected/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64-kube-api-access-djrr6\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:59:19.103521 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.103297    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxnf7\" (UniqueName: \"kubernetes.io/projected/feab9c5d-7711-494f-9676-42a1aa8b183d-kube-api-access-fxnf7\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:59:19.581451 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.581416    2556 generic.go:356] "Generic (PLEG): container finished" podID="9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64" containerID="c3d3aaf2aa08143cd35992c14c8f2c5ad13df9bb10792b335afbfdace9077e2e" exitCode=0
Aug 05 17:59:19.581838 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.581460    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"
Aug 05 17:59:19.581838 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.581500    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h" event={"ID":"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64","Type":"ContainerDied","Data":"c3d3aaf2aa08143cd35992c14c8f2c5ad13df9bb10792b335afbfdace9077e2e"}
Aug 05 17:59:19.581838 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.581550    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h" event={"ID":"9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64","Type":"ContainerDied","Data":"4cc6982a7c9e89fec5d673a9b9f65504c8153902c068ac4940e4b5f57ce4e7da"}
Aug 05 17:59:19.581838 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.581572    2556 scope.go:117] "RemoveContainer" containerID="c3d3aaf2aa08143cd35992c14c8f2c5ad13df9bb10792b335afbfdace9077e2e"
Aug 05 17:59:19.583081 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.583056    2556 generic.go:356] "Generic (PLEG): container finished" podID="feab9c5d-7711-494f-9676-42a1aa8b183d" containerID="e02e2500a08708412a12916f15ba6607938b1ce51f2974361c38ed091a98dbe4" exitCode=0
Aug 05 17:59:19.583203 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.583107    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"
Aug 05 17:59:19.583203 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.583165    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl" event={"ID":"feab9c5d-7711-494f-9676-42a1aa8b183d","Type":"ContainerDied","Data":"e02e2500a08708412a12916f15ba6607938b1ce51f2974361c38ed091a98dbe4"}
Aug 05 17:59:19.583334 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.583220    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl" event={"ID":"feab9c5d-7711-494f-9676-42a1aa8b183d","Type":"ContainerDied","Data":"4460b5d67a2dcc7abf930ab2a3add85c8a34c4195b70666c60ff4c5f55a2e1f4"}
Aug 05 17:59:19.584607 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.584583    2556 generic.go:356] "Generic (PLEG): container finished" podID="ece7846d-612a-4f8b-99ba-c532c4f55010" containerID="1d5cf1310f70bf62e80f329b3804bb1e4c7266fa2b57835c17f55f8c8826e198" exitCode=0
Aug 05 17:59:19.584713 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.584624    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9" event={"ID":"ece7846d-612a-4f8b-99ba-c532c4f55010","Type":"ContainerDied","Data":"1d5cf1310f70bf62e80f329b3804bb1e4c7266fa2b57835c17f55f8c8826e198"}
Aug 05 17:59:19.584713 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.584638    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"
Aug 05 17:59:19.584713 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.584661    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9" event={"ID":"ece7846d-612a-4f8b-99ba-c532c4f55010","Type":"ContainerDied","Data":"fa07b4c60916e61b925b07c8aa03cca4b5ccf72368f69f88c2a59dd77e02c472"}
Aug 05 17:59:19.591452 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.591437    2556 scope.go:117] "RemoveContainer" containerID="c3d3aaf2aa08143cd35992c14c8f2c5ad13df9bb10792b335afbfdace9077e2e"
Aug 05 17:59:19.591688 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:59:19.591667    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3d3aaf2aa08143cd35992c14c8f2c5ad13df9bb10792b335afbfdace9077e2e\": container with ID starting with c3d3aaf2aa08143cd35992c14c8f2c5ad13df9bb10792b335afbfdace9077e2e not found: ID does not exist" containerID="c3d3aaf2aa08143cd35992c14c8f2c5ad13df9bb10792b335afbfdace9077e2e"
Aug 05 17:59:19.591758 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.591696    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3d3aaf2aa08143cd35992c14c8f2c5ad13df9bb10792b335afbfdace9077e2e"} err="failed to get container status \"c3d3aaf2aa08143cd35992c14c8f2c5ad13df9bb10792b335afbfdace9077e2e\": rpc error: code = NotFound desc = could not find container \"c3d3aaf2aa08143cd35992c14c8f2c5ad13df9bb10792b335afbfdace9077e2e\": container with ID starting with c3d3aaf2aa08143cd35992c14c8f2c5ad13df9bb10792b335afbfdace9077e2e not found: ID does not exist"
Aug 05 17:59:19.591758 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.591711    2556 scope.go:117] "RemoveContainer" containerID="e02e2500a08708412a12916f15ba6607938b1ce51f2974361c38ed091a98dbe4"
Aug 05 17:59:19.598950 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.598931    2556 scope.go:117] "RemoveContainer" containerID="e02e2500a08708412a12916f15ba6607938b1ce51f2974361c38ed091a98dbe4"
Aug 05 17:59:19.599247 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:59:19.599140    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e02e2500a08708412a12916f15ba6607938b1ce51f2974361c38ed091a98dbe4\": container with ID starting with e02e2500a08708412a12916f15ba6607938b1ce51f2974361c38ed091a98dbe4 not found: ID does not exist" containerID="e02e2500a08708412a12916f15ba6607938b1ce51f2974361c38ed091a98dbe4"
Aug 05 17:59:19.599309 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.599254    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e02e2500a08708412a12916f15ba6607938b1ce51f2974361c38ed091a98dbe4"} err="failed to get container status \"e02e2500a08708412a12916f15ba6607938b1ce51f2974361c38ed091a98dbe4\": rpc error: code = NotFound desc = could not find container \"e02e2500a08708412a12916f15ba6607938b1ce51f2974361c38ed091a98dbe4\": container with ID starting with e02e2500a08708412a12916f15ba6607938b1ce51f2974361c38ed091a98dbe4 not found: ID does not exist"
Aug 05 17:59:19.599309 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.599270    2556 scope.go:117] "RemoveContainer" containerID="1d5cf1310f70bf62e80f329b3804bb1e4c7266fa2b57835c17f55f8c8826e198"
Aug 05 17:59:19.603199 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.603179    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"]
Aug 05 17:59:19.607471 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.607270    2556 scope.go:117] "RemoveContainer" containerID="1d5cf1310f70bf62e80f329b3804bb1e4c7266fa2b57835c17f55f8c8826e198"
Aug 05 17:59:19.607573 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:59:19.607532    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d5cf1310f70bf62e80f329b3804bb1e4c7266fa2b57835c17f55f8c8826e198\": container with ID starting with 1d5cf1310f70bf62e80f329b3804bb1e4c7266fa2b57835c17f55f8c8826e198 not found: ID does not exist" containerID="1d5cf1310f70bf62e80f329b3804bb1e4c7266fa2b57835c17f55f8c8826e198"
Aug 05 17:59:19.607645 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.607574    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d5cf1310f70bf62e80f329b3804bb1e4c7266fa2b57835c17f55f8c8826e198"} err="failed to get container status \"1d5cf1310f70bf62e80f329b3804bb1e4c7266fa2b57835c17f55f8c8826e198\": rpc error: code = NotFound desc = could not find container \"1d5cf1310f70bf62e80f329b3804bb1e4c7266fa2b57835c17f55f8c8826e198\": container with ID starting with 1d5cf1310f70bf62e80f329b3804bb1e4c7266fa2b57835c17f55f8c8826e198 not found: ID does not exist"
Aug 05 17:59:19.607707 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.607647    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b60f38ad-74f6d9769f-z799h"]
Aug 05 17:59:19.617813 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.617794    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"]
Aug 05 17:59:19.621763 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.621742    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b60f38ad-d5f85ccfd-qbwwl"]
Aug 05 17:59:19.633754 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.633734    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"]
Aug 05 17:59:19.639574 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:19.639554    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b60f38ad-67c456b896-6srs9"]
Aug 05 17:59:21.096321 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:21.096288    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64" path="/var/lib/kubelet/pods/9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64/volumes"
Aug 05 17:59:21.096825 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:21.096804    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ece7846d-612a-4f8b-99ba-c532c4f55010" path="/var/lib/kubelet/pods/ece7846d-612a-4f8b-99ba-c532c4f55010/volumes"
Aug 05 17:59:21.097205 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:21.097187    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="feab9c5d-7711-494f-9676-42a1aa8b183d" path="/var/lib/kubelet/pods/feab9c5d-7711-494f-9676-42a1aa8b183d/volumes"
Aug 05 17:59:27.010610 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:27.010572    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"]
Aug 05 17:59:42.029961 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.029921    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc" podUID="32d3d762-856b-4942-9b2c-ea513157de8e" containerName="main" containerID="cri-o://48e93b888c6f4f967d25982d0e904077aab3c68d56881a58a5b3152ff9e44d1d" gracePeriod=45
Aug 05 17:59:42.276452 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.276427    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:59:42.395523 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.395448    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlgtc\" (UniqueName: \"kubernetes.io/projected/32d3d762-856b-4942-9b2c-ea513157de8e-kube-api-access-zlgtc\") pod \"32d3d762-856b-4942-9b2c-ea513157de8e\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") "
Aug 05 17:59:42.395523 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.395482    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-model-cache\") pod \"32d3d762-856b-4942-9b2c-ea513157de8e\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") "
Aug 05 17:59:42.395523 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.395512    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-tmp-dir\") pod \"32d3d762-856b-4942-9b2c-ea513157de8e\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") "
Aug 05 17:59:42.395791 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.395530    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-dshm\") pod \"32d3d762-856b-4942-9b2c-ea513157de8e\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") "
Aug 05 17:59:42.395791 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.395583    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32d3d762-856b-4942-9b2c-ea513157de8e-tls-certs\") pod \"32d3d762-856b-4942-9b2c-ea513157de8e\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") "
Aug 05 17:59:42.395791 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.395606    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-home\") pod \"32d3d762-856b-4942-9b2c-ea513157de8e\" (UID: \"32d3d762-856b-4942-9b2c-ea513157de8e\") "
Aug 05 17:59:42.396035 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.395988    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "32d3d762-856b-4942-9b2c-ea513157de8e" (UID: "32d3d762-856b-4942-9b2c-ea513157de8e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:59:42.396261 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.396226    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-model-cache" (OuterVolumeSpecName: "model-cache") pod "32d3d762-856b-4942-9b2c-ea513157de8e" (UID: "32d3d762-856b-4942-9b2c-ea513157de8e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:59:42.396546 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.396523    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-home" (OuterVolumeSpecName: "home") pod "32d3d762-856b-4942-9b2c-ea513157de8e" (UID: "32d3d762-856b-4942-9b2c-ea513157de8e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:59:42.398250 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.396938    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-model-cache\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:59:42.398250 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.396968    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-tmp-dir\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:59:42.398250 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.397004    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-home\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:59:42.398947 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.398891    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32d3d762-856b-4942-9b2c-ea513157de8e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "32d3d762-856b-4942-9b2c-ea513157de8e" (UID: "32d3d762-856b-4942-9b2c-ea513157de8e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:59:42.399168 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.399038    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32d3d762-856b-4942-9b2c-ea513157de8e-kube-api-access-zlgtc" (OuterVolumeSpecName: "kube-api-access-zlgtc") pod "32d3d762-856b-4942-9b2c-ea513157de8e" (UID: "32d3d762-856b-4942-9b2c-ea513157de8e"). InnerVolumeSpecName "kube-api-access-zlgtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:59:42.403997 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.403975    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-dshm" (OuterVolumeSpecName: "dshm") pod "32d3d762-856b-4942-9b2c-ea513157de8e" (UID: "32d3d762-856b-4942-9b2c-ea513157de8e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:59:42.498127 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.498092    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlgtc\" (UniqueName: \"kubernetes.io/projected/32d3d762-856b-4942-9b2c-ea513157de8e-kube-api-access-zlgtc\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:59:42.498127 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.498125    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32d3d762-856b-4942-9b2c-ea513157de8e-dshm\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:59:42.498127 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.498136    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32d3d762-856b-4942-9b2c-ea513157de8e-tls-certs\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 17:59:42.695026 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.694942    2556 generic.go:356] "Generic (PLEG): container finished" podID="32d3d762-856b-4942-9b2c-ea513157de8e" containerID="48e93b888c6f4f967d25982d0e904077aab3c68d56881a58a5b3152ff9e44d1d" exitCode=0
Aug 05 17:59:42.695026 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.694996    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"
Aug 05 17:59:42.695026 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.695012    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc" event={"ID":"32d3d762-856b-4942-9b2c-ea513157de8e","Type":"ContainerDied","Data":"48e93b888c6f4f967d25982d0e904077aab3c68d56881a58a5b3152ff9e44d1d"}
Aug 05 17:59:42.695293 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.695041    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc" event={"ID":"32d3d762-856b-4942-9b2c-ea513157de8e","Type":"ContainerDied","Data":"061d11ee62795631e1b279876eb41b7a774d1f0860bf4b9eb902fe09959b11e0"}
Aug 05 17:59:42.695293 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.695057    2556 scope.go:117] "RemoveContainer" containerID="48e93b888c6f4f967d25982d0e904077aab3c68d56881a58a5b3152ff9e44d1d"
Aug 05 17:59:42.703903 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.703887    2556 scope.go:117] "RemoveContainer" containerID="48e93b888c6f4f967d25982d0e904077aab3c68d56881a58a5b3152ff9e44d1d"
Aug 05 17:59:42.704145 ip-10-0-141-37 kubenswrapper[2556]: E0805 17:59:42.704128    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48e93b888c6f4f967d25982d0e904077aab3c68d56881a58a5b3152ff9e44d1d\": container with ID starting with 48e93b888c6f4f967d25982d0e904077aab3c68d56881a58a5b3152ff9e44d1d not found: ID does not exist" containerID="48e93b888c6f4f967d25982d0e904077aab3c68d56881a58a5b3152ff9e44d1d"
Aug 05 17:59:42.704280 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.704169    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e93b888c6f4f967d25982d0e904077aab3c68d56881a58a5b3152ff9e44d1d"} err="failed to get container status \"48e93b888c6f4f967d25982d0e904077aab3c68d56881a58a5b3152ff9e44d1d\": rpc error: code = NotFound desc = could not find container \"48e93b888c6f4f967d25982d0e904077aab3c68d56881a58a5b3152ff9e44d1d\": container with ID starting with 48e93b888c6f4f967d25982d0e904077aab3c68d56881a58a5b3152ff9e44d1d not found: ID does not exist"
Aug 05 17:59:42.715742 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.715717    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"]
Aug 05 17:59:42.720731 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:42.720712    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-b60f38ad/e2e-lifecycle-model-b60f38ad-kserve-dc7c877f5-jh9hc"]
Aug 05 17:59:43.097986 ip-10-0-141-37 kubenswrapper[2556]: I0805 17:59:43.097950    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32d3d762-856b-4942-9b2c-ea513157de8e" path="/var/lib/kubelet/pods/32d3d762-856b-4942-9b2c-ea513157de8e/volumes"
Aug 05 18:01:20.190744 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.190710    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"]
Aug 05 18:01:20.191171 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.191053    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64" containerName="maas-api"
Aug 05 18:01:20.191171 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.191064    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64" containerName="maas-api"
Aug 05 18:01:20.191171 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.191081    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32d3d762-856b-4942-9b2c-ea513157de8e" containerName="main"
Aug 05 18:01:20.191171 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.191086    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="32d3d762-856b-4942-9b2c-ea513157de8e" containerName="main"
Aug 05 18:01:20.191171 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.191095    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feab9c5d-7711-494f-9676-42a1aa8b183d" containerName="payload-processing"
Aug 05 18:01:20.191171 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.191101    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="feab9c5d-7711-494f-9676-42a1aa8b183d" containerName="payload-processing"
Aug 05 18:01:20.191171 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.191112    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece7846d-612a-4f8b-99ba-c532c4f55010" containerName="payload-pre-processing"
Aug 05 18:01:20.191171 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.191117    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece7846d-612a-4f8b-99ba-c532c4f55010" containerName="payload-pre-processing"
Aug 05 18:01:20.191424 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.191194    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="feab9c5d-7711-494f-9676-42a1aa8b183d" containerName="payload-processing"
Aug 05 18:01:20.191424 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.191203    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c1cfa2e-4b89-4cd9-85ff-b575aa47bd64" containerName="maas-api"
Aug 05 18:01:20.191424 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.191209    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece7846d-612a-4f8b-99ba-c532c4f55010" containerName="payload-pre-processing"
Aug 05 18:01:20.191424 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.191215    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="32d3d762-856b-4942-9b2c-ea513157de8e" containerName="main"
Aug 05 18:01:20.194375 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.194356    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"
Aug 05 18:01:20.197573 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.197552    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-19f4b3-serving-cert\""
Aug 05 18:01:20.197708 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.197624    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ql6kj\""
Aug 05 18:01:20.216916 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.216890    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"]
Aug 05 18:01:20.266917 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.266885    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw788\" (UniqueName: \"kubernetes.io/projected/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-kube-api-access-pw788\") pod \"maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t\" (UID: \"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"
Aug 05 18:01:20.267054 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.266929    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-maas-api-tls\") pod \"maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t\" (UID: \"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"
Aug 05 18:01:20.267054 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.266951    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t\" (UID: \"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"
Aug 05 18:01:20.368291 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.368258    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw788\" (UniqueName: \"kubernetes.io/projected/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-kube-api-access-pw788\") pod \"maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t\" (UID: \"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"
Aug 05 18:01:20.368462 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.368303    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-maas-api-tls\") pod \"maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t\" (UID: \"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"
Aug 05 18:01:20.368462 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.368329    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t\" (UID: \"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"
Aug 05 18:01:20.368934 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.368911    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t\" (UID: \"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"
Aug 05 18:01:20.370779 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.370758    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-maas-api-tls\") pod \"maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t\" (UID: \"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"
Aug 05 18:01:20.377259 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.377236    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw788\" (UniqueName: \"kubernetes.io/projected/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-kube-api-access-pw788\") pod \"maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t\" (UID: \"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"
Aug 05 18:01:20.505616 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.505582    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"
Aug 05 18:01:20.645008 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.644985    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"]
Aug 05 18:01:20.646719 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:01:20.646692    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b1d7850_d389_4a17_a99b_e7b2a1e4b85f.slice/crio-abb788f83ac466a1a8b676d0143f392032c8600984e875097b50d38cdf6faa12 WatchSource:0}: Error finding container abb788f83ac466a1a8b676d0143f392032c8600984e875097b50d38cdf6faa12: Status 404 returned error can't find the container with id abb788f83ac466a1a8b676d0143f392032c8600984e875097b50d38cdf6faa12
Aug 05 18:01:20.891820 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.891747    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"]
Aug 05 18:01:20.896304 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.896285    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"
Aug 05 18:01:20.898835 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.898816    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-19f4b3-dockercfg-7xwjw\""
Aug 05 18:01:20.898918 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.898830    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-19f4b3\""
Aug 05 18:01:20.904783 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.904761    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"]
Aug 05 18:01:20.916038 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.916015    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"]
Aug 05 18:01:20.923171 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.919837    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"
Aug 05 18:01:20.929919 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.929891    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"]
Aug 05 18:01:20.974125 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.974100    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqcx4\" (UniqueName: \"kubernetes.io/projected/7b76ea36-25d6-464a-acb3-379bec538d3e-kube-api-access-sqcx4\") pod \"payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p\" (UID: \"7b76ea36-25d6-464a-acb3-379bec538d3e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"
Aug 05 18:01:20.974252 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.974134    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhmqh\" (UniqueName: \"kubernetes.io/projected/030d012c-030f-4eda-b220-f73c3956a041-kube-api-access-lhmqh\") pod \"payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn\" (UID: \"030d012c-030f-4eda-b220-f73c3956a041\") " pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"
Aug 05 18:01:20.974252 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.974206    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/030d012c-030f-4eda-b220-f73c3956a041-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn\" (UID: \"030d012c-030f-4eda-b220-f73c3956a041\") " pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"
Aug 05 18:01:20.974361 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:20.974249    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b76ea36-25d6-464a-acb3-379bec538d3e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p\" (UID: \"7b76ea36-25d6-464a-acb3-379bec538d3e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"
Aug 05 18:01:21.075638 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.075605    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqcx4\" (UniqueName: \"kubernetes.io/projected/7b76ea36-25d6-464a-acb3-379bec538d3e-kube-api-access-sqcx4\") pod \"payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p\" (UID: \"7b76ea36-25d6-464a-acb3-379bec538d3e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"
Aug 05 18:01:21.075751 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.075656    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhmqh\" (UniqueName: \"kubernetes.io/projected/030d012c-030f-4eda-b220-f73c3956a041-kube-api-access-lhmqh\") pod \"payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn\" (UID: \"030d012c-030f-4eda-b220-f73c3956a041\") " pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"
Aug 05 18:01:21.075751 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.075687    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/030d012c-030f-4eda-b220-f73c3956a041-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn\" (UID: \"030d012c-030f-4eda-b220-f73c3956a041\") " pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"
Aug 05 18:01:21.075751 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.075715    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b76ea36-25d6-464a-acb3-379bec538d3e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p\" (UID: \"7b76ea36-25d6-464a-acb3-379bec538d3e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"
Aug 05 18:01:21.076460 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.076439    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b76ea36-25d6-464a-acb3-379bec538d3e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p\" (UID: \"7b76ea36-25d6-464a-acb3-379bec538d3e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"
Aug 05 18:01:21.076460 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.076450    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/030d012c-030f-4eda-b220-f73c3956a041-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn\" (UID: \"030d012c-030f-4eda-b220-f73c3956a041\") " pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"
Aug 05 18:01:21.083713 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.083693    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqcx4\" (UniqueName: \"kubernetes.io/projected/7b76ea36-25d6-464a-acb3-379bec538d3e-kube-api-access-sqcx4\") pod \"payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p\" (UID: \"7b76ea36-25d6-464a-acb3-379bec538d3e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"
Aug 05 18:01:21.083890 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.083870    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhmqh\" (UniqueName: \"kubernetes.io/projected/030d012c-030f-4eda-b220-f73c3956a041-kube-api-access-lhmqh\") pod \"payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn\" (UID: \"030d012c-030f-4eda-b220-f73c3956a041\") " pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"
Aug 05 18:01:21.143999 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.143926    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t" event={"ID":"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f","Type":"ContainerStarted","Data":"86d27cb11574021c71dc9cc3fc264a8e2a8289b22d726d30b562399f523789b6"}
Aug 05 18:01:21.143999 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.143977    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t" event={"ID":"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f","Type":"ContainerStarted","Data":"abb788f83ac466a1a8b676d0143f392032c8600984e875097b50d38cdf6faa12"}
Aug 05 18:01:21.144223 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.144119    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"
Aug 05 18:01:21.160878 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.160831    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t" podStartSLOduration=0.817574755 podStartE2EDuration="1.16081764s" totalImagesPullingTime="343.242885ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:01:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:01:21.15966401 +0000 UTC m=+2494.577978787" watchObservedRunningTime="2026-08-05 18:01:21.16081764 +0000 UTC m=+2494.579132397"
Aug 05 18:01:21.207951 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.207924    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"
Aug 05 18:01:21.236356 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.236329    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"
Aug 05 18:01:21.358915 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.358887    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"]
Aug 05 18:01:21.362921 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:01:21.362891    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b76ea36_25d6_464a_acb3_379bec538d3e.slice/crio-9a4b6f90c224e5903c4f9d96adc26e79bbc7ee3bdcfe270a3c6b8c1a0b818b9c WatchSource:0}: Error finding container 9a4b6f90c224e5903c4f9d96adc26e79bbc7ee3bdcfe270a3c6b8c1a0b818b9c: Status 404 returned error can't find the container with id 9a4b6f90c224e5903c4f9d96adc26e79bbc7ee3bdcfe270a3c6b8c1a0b818b9c
Aug 05 18:01:21.385774 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:21.385748    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"]
Aug 05 18:01:21.387239 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:01:21.387216    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod030d012c_030f_4eda_b220_f73c3956a041.slice/crio-aae27db6f45996c599681b8b909c3a8449a41bc4aa9d02dec03908838d0bbf9f WatchSource:0}: Error finding container aae27db6f45996c599681b8b909c3a8449a41bc4aa9d02dec03908838d0bbf9f: Status 404 returned error can't find the container with id aae27db6f45996c599681b8b909c3a8449a41bc4aa9d02dec03908838d0bbf9f
Aug 05 18:01:22.150432 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:22.150399    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn" event={"ID":"030d012c-030f-4eda-b220-f73c3956a041","Type":"ContainerStarted","Data":"8e4bc6498a125ee1939badd476e4e7f0421a04a11ae34f81c132343acf3de953"}
Aug 05 18:01:22.150649 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:22.150458    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn" event={"ID":"030d012c-030f-4eda-b220-f73c3956a041","Type":"ContainerStarted","Data":"aae27db6f45996c599681b8b909c3a8449a41bc4aa9d02dec03908838d0bbf9f"}
Aug 05 18:01:22.150649 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:22.150489    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"
Aug 05 18:01:22.151873 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:22.151847    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p" event={"ID":"7b76ea36-25d6-464a-acb3-379bec538d3e","Type":"ContainerStarted","Data":"f9d1fa135f3bcc084e86c13b18339142fc5ec3c71e901c4d0ee23f38cc0b0abf"}
Aug 05 18:01:22.152002 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:22.151894    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p" event={"ID":"7b76ea36-25d6-464a-acb3-379bec538d3e","Type":"ContainerStarted","Data":"9a4b6f90c224e5903c4f9d96adc26e79bbc7ee3bdcfe270a3c6b8c1a0b818b9c"}
Aug 05 18:01:22.166950 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:22.166909    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn" podStartSLOduration=2.166895183 podStartE2EDuration="2.166895183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:01:22.165319106 +0000 UTC m=+2495.583633863" watchObservedRunningTime="2026-08-05 18:01:22.166895183 +0000 UTC m=+2495.585209940"
Aug 05 18:01:22.180316 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:22.180274    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p" podStartSLOduration=2.180262686 podStartE2EDuration="2.180262686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:01:22.178971497 +0000 UTC m=+2495.597286293" watchObservedRunningTime="2026-08-05 18:01:22.180262686 +0000 UTC m=+2495.598577443"
Aug 05 18:01:23.158496 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:23.158462    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"
Aug 05 18:01:27.157919 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:27.157891    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"
Aug 05 18:01:33.160193 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:33.160133    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"
Aug 05 18:01:34.166956 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:34.166929    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"
Aug 05 18:01:48.734567 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:48.734526    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"]
Aug 05 18:01:48.737336 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:48.737318    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"
Aug 05 18:01:48.739842 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:48.739819    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-76d454-serving-cert\""
Aug 05 18:01:48.748553 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:48.748524    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"]
Aug 05 18:01:48.904935 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:48.904900    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl76q\" (UniqueName: \"kubernetes.io/projected/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-kube-api-access-nl76q\") pod \"maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq\" (UID: \"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"
Aug 05 18:01:48.905111 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:48.904965    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-maas-api-tls\") pod \"maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq\" (UID: \"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"
Aug 05 18:01:48.905111 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:48.905013    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq\" (UID: \"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"
Aug 05 18:01:49.005749 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.005672    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl76q\" (UniqueName: \"kubernetes.io/projected/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-kube-api-access-nl76q\") pod \"maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq\" (UID: \"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"
Aug 05 18:01:49.005749 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.005730    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-maas-api-tls\") pod \"maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq\" (UID: \"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"
Aug 05 18:01:49.005943 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.005765    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq\" (UID: \"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"
Aug 05 18:01:49.006471 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.006449    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq\" (UID: \"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"
Aug 05 18:01:49.008085 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.008055    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-maas-api-tls\") pod \"maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq\" (UID: \"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"
Aug 05 18:01:49.013909 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.013885    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl76q\" (UniqueName: \"kubernetes.io/projected/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-kube-api-access-nl76q\") pod \"maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq\" (UID: \"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"
Aug 05 18:01:49.048852 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.048828    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"
Aug 05 18:01:49.188332 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.188294    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"]
Aug 05 18:01:49.191115 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:01:49.191084    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58f4103f_fc3f_4cc1_b7a8_9a2c0356fdf9.slice/crio-50ec4304626e3fe7aa3622c8a69f4e7897501e30bb0359f0b29cf61583a7dce3 WatchSource:0}: Error finding container 50ec4304626e3fe7aa3622c8a69f4e7897501e30bb0359f0b29cf61583a7dce3: Status 404 returned error can't find the container with id 50ec4304626e3fe7aa3622c8a69f4e7897501e30bb0359f0b29cf61583a7dce3
Aug 05 18:01:49.283134 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.283056    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq" event={"ID":"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9","Type":"ContainerStarted","Data":"50ec4304626e3fe7aa3622c8a69f4e7897501e30bb0359f0b29cf61583a7dce3"}
Aug 05 18:01:49.395935 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.395901    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"]
Aug 05 18:01:49.406372 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.406339    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"
Aug 05 18:01:49.408570 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.408544    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"]
Aug 05 18:01:49.409250 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.409088    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-76d454-dockercfg-w466r\""
Aug 05 18:01:49.409250 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.409108    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-76d454\""
Aug 05 18:01:49.509591 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.509560    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/353f8056-3519-448f-89bc-9ae4bf841932-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc\" (UID: \"353f8056-3519-448f-89bc-9ae4bf841932\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"
Aug 05 18:01:49.509770 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.509714    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwtq8\" (UniqueName: \"kubernetes.io/projected/353f8056-3519-448f-89bc-9ae4bf841932-kube-api-access-fwtq8\") pod \"payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc\" (UID: \"353f8056-3519-448f-89bc-9ae4bf841932\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"
Aug 05 18:01:49.610599 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.610511    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwtq8\" (UniqueName: \"kubernetes.io/projected/353f8056-3519-448f-89bc-9ae4bf841932-kube-api-access-fwtq8\") pod \"payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc\" (UID: \"353f8056-3519-448f-89bc-9ae4bf841932\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"
Aug 05 18:01:49.610599 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.610568    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/353f8056-3519-448f-89bc-9ae4bf841932-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc\" (UID: \"353f8056-3519-448f-89bc-9ae4bf841932\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"
Aug 05 18:01:49.611466 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.611439    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/353f8056-3519-448f-89bc-9ae4bf841932-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc\" (UID: \"353f8056-3519-448f-89bc-9ae4bf841932\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"
Aug 05 18:01:49.620132 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.620103    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwtq8\" (UniqueName: \"kubernetes.io/projected/353f8056-3519-448f-89bc-9ae4bf841932-kube-api-access-fwtq8\") pod \"payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc\" (UID: \"353f8056-3519-448f-89bc-9ae4bf841932\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"
Aug 05 18:01:49.720144 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:49.720117    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"
Aug 05 18:01:50.055509 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:50.055457    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"]
Aug 05 18:01:50.057709 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:01:50.057674    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353f8056_3519_448f_89bc_9ae4bf841932.slice/crio-2fd83fcc3f1ee8591f9996e4242335bec4adf3b352fff0cfada05f1d4914c9fd WatchSource:0}: Error finding container 2fd83fcc3f1ee8591f9996e4242335bec4adf3b352fff0cfada05f1d4914c9fd: Status 404 returned error can't find the container with id 2fd83fcc3f1ee8591f9996e4242335bec4adf3b352fff0cfada05f1d4914c9fd
Aug 05 18:01:50.289534 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:50.289498    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc" event={"ID":"353f8056-3519-448f-89bc-9ae4bf841932","Type":"ContainerStarted","Data":"783705bf6ae9da146d20989546ec8578337e00196fed9f3f0fbec86f05ac0127"}
Aug 05 18:01:50.289727 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:50.289560    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc" event={"ID":"353f8056-3519-448f-89bc-9ae4bf841932","Type":"ContainerStarted","Data":"2fd83fcc3f1ee8591f9996e4242335bec4adf3b352fff0cfada05f1d4914c9fd"}
Aug 05 18:01:50.289727 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:50.289602    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"
Aug 05 18:01:50.306018 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:50.305928    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc" podStartSLOduration=1.305913095 podStartE2EDuration="1.305913095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:01:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:01:50.304643637 +0000 UTC m=+2523.722958394" watchObservedRunningTime="2026-08-05 18:01:50.305913095 +0000 UTC m=+2523.724227852"
Aug 05 18:01:51.295825 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:51.295793    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq" event={"ID":"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9","Type":"ContainerStarted","Data":"ad44322d21560a79f4d700829c9936c55941bdcdd65e43cff925b0b7cdce3b8b"}
Aug 05 18:01:51.296294 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:51.295870    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"
Aug 05 18:01:51.314453 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:51.314400    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq" podStartSLOduration=1.975591771 podStartE2EDuration="3.314384855s" totalImagesPullingTime="1.338793084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:01:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:01:51.312233967 +0000 UTC m=+2524.730548745" watchObservedRunningTime="2026-08-05 18:01:51.314384855 +0000 UTC m=+2524.732699646"
Aug 05 18:01:57.305526 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:01:57.305498    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"
Aug 05 18:02:01.298480 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:01.298452    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"
Aug 05 18:02:17.460313 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.460280    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"]
Aug 05 18:02:17.464916 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.464889    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.467514 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.467492    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-19f4b3\"/\"openshift-service-ca.crt\""
Aug 05 18:02:17.467648 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.467517    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-19f4b3\"/\"test-model-19f4b3-kserve-self-signed-certs\""
Aug 05 18:02:17.468719 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.468694    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-19f4b3\"/\"kube-root-ca.crt\""
Aug 05 18:02:17.468989 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.468971    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-19f4b3\"/\"default-dockercfg-zdrv2\""
Aug 05 18:02:17.471299 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.471275    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"]
Aug 05 18:02:17.551675 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.551645    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hs69\" (UniqueName: \"kubernetes.io/projected/2fefa718-7a57-4888-ba8c-6028e617dc8e-kube-api-access-8hs69\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.551850 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.551712    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-home\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.551850 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.551767    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-model-cache\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.551850 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.551799    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-dshm\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.551850 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.551826    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fefa718-7a57-4888-ba8c-6028e617dc8e-tls-certs\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.552015 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.551849    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-tmp-dir\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.652500 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.652467    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-home\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.652654 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.652514    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-model-cache\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.652654 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.652542    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-dshm\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.652654 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.652569    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fefa718-7a57-4888-ba8c-6028e617dc8e-tls-certs\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.652654 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.652593    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-tmp-dir\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.652845 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.652738    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hs69\" (UniqueName: \"kubernetes.io/projected/2fefa718-7a57-4888-ba8c-6028e617dc8e-kube-api-access-8hs69\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.652932 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.652913    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-home\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.653187 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.652930    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-tmp-dir\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.653187 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.653007    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-model-cache\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.654755 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.654734    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-dshm\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.655099 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.655084    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fefa718-7a57-4888-ba8c-6028e617dc8e-tls-certs\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.660862 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.660839    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hs69\" (UniqueName: \"kubernetes.io/projected/2fefa718-7a57-4888-ba8c-6028e617dc8e-kube-api-access-8hs69\") pod \"test-model-19f4b3-kserve-85f6c68cff-tcpq4\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") " pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.775591 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.775543    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:17.905715 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:17.905683    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"]
Aug 05 18:02:17.906845 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:02:17.906807    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fefa718_7a57_4888_ba8c_6028e617dc8e.slice/crio-d5712b382ad3e7b4359636c224b35d79550c1f98fd00321159d046b9725b8b88 WatchSource:0}: Error finding container d5712b382ad3e7b4359636c224b35d79550c1f98fd00321159d046b9725b8b88: Status 404 returned error can't find the container with id d5712b382ad3e7b4359636c224b35d79550c1f98fd00321159d046b9725b8b88
Aug 05 18:02:18.427442 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:18.427405    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4" event={"ID":"2fefa718-7a57-4888-ba8c-6028e617dc8e","Type":"ContainerStarted","Data":"9b54108cb8fbd6f7b5e23b8e1e13bcf37f3954c052a241b47fef279821bd553d"}
Aug 05 18:02:18.427442 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:18.427443    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4" event={"ID":"2fefa718-7a57-4888-ba8c-6028e617dc8e","Type":"ContainerStarted","Data":"d5712b382ad3e7b4359636c224b35d79550c1f98fd00321159d046b9725b8b88"}
Aug 05 18:02:18.444041 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:18.443992    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4" podStartSLOduration=1.44397731 podStartE2EDuration="1.44397731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:02:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:02:18.442740531 +0000 UTC m=+2551.861055287" watchObservedRunningTime="2026-08-05 18:02:18.44397731 +0000 UTC m=+2551.862292067"
Aug 05 18:02:27.776664 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:27.776631    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:27.777225 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:27.776708    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:27.789477 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:27.789452    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:02:28.486728 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:02:28.486701    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:03:32.233679 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:32.233598    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"]
Aug 05 18:03:43.792441 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:43.792412    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"]
Aug 05 18:03:43.792965 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:43.792652    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t" podUID="5b1d7850-d389-4a17-a99b-e7b2a1e4b85f" containerName="maas-api" containerID="cri-o://86d27cb11574021c71dc9cc3fc264a8e2a8289b22d726d30b562399f523789b6" gracePeriod=30
Aug 05 18:03:43.848315 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:43.848285    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"]
Aug 05 18:03:43.848505 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:43.848483    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn" podUID="030d012c-030f-4eda-b220-f73c3956a041" containerName="payload-processing" containerID="cri-o://8e4bc6498a125ee1939badd476e4e7f0421a04a11ae34f81c132343acf3de953" gracePeriod=30
Aug 05 18:03:43.854687 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:43.854660    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"]
Aug 05 18:03:43.854945 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:43.854886    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p" podUID="7b76ea36-25d6-464a-acb3-379bec538d3e" containerName="payload-pre-processing" containerID="cri-o://f9d1fa135f3bcc084e86c13b18339142fc5ec3c71e901c4d0ee23f38cc0b0abf" gracePeriod=30
Aug 05 18:03:44.159391 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.159361    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"
Aug 05 18:03:44.162554 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.162533    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"
Aug 05 18:03:44.249852 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.249816    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"
Aug 05 18:03:44.257472 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.257447    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqcx4\" (UniqueName: \"kubernetes.io/projected/7b76ea36-25d6-464a-acb3-379bec538d3e-kube-api-access-sqcx4\") pod \"7b76ea36-25d6-464a-acb3-379bec538d3e\" (UID: \"7b76ea36-25d6-464a-acb3-379bec538d3e\") "
Aug 05 18:03:44.257607 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.257486    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b76ea36-25d6-464a-acb3-379bec538d3e-plugins-config-volume\") pod \"7b76ea36-25d6-464a-acb3-379bec538d3e\" (UID: \"7b76ea36-25d6-464a-acb3-379bec538d3e\") "
Aug 05 18:03:44.257607 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.257582    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/030d012c-030f-4eda-b220-f73c3956a041-plugins-config-volume\") pod \"030d012c-030f-4eda-b220-f73c3956a041\" (UID: \"030d012c-030f-4eda-b220-f73c3956a041\") "
Aug 05 18:03:44.257727 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.257621    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhmqh\" (UniqueName: \"kubernetes.io/projected/030d012c-030f-4eda-b220-f73c3956a041-kube-api-access-lhmqh\") pod \"030d012c-030f-4eda-b220-f73c3956a041\" (UID: \"030d012c-030f-4eda-b220-f73c3956a041\") "
Aug 05 18:03:44.258091 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.258051    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/030d012c-030f-4eda-b220-f73c3956a041-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "030d012c-030f-4eda-b220-f73c3956a041" (UID: "030d012c-030f-4eda-b220-f73c3956a041"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:03:44.258091 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.258076    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b76ea36-25d6-464a-acb3-379bec538d3e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7b76ea36-25d6-464a-acb3-379bec538d3e" (UID: "7b76ea36-25d6-464a-acb3-379bec538d3e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:03:44.259537 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.259503    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b76ea36-25d6-464a-acb3-379bec538d3e-kube-api-access-sqcx4" (OuterVolumeSpecName: "kube-api-access-sqcx4") pod "7b76ea36-25d6-464a-acb3-379bec538d3e" (UID: "7b76ea36-25d6-464a-acb3-379bec538d3e"). InnerVolumeSpecName "kube-api-access-sqcx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:03:44.259887 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.259863    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/030d012c-030f-4eda-b220-f73c3956a041-kube-api-access-lhmqh" (OuterVolumeSpecName: "kube-api-access-lhmqh") pod "030d012c-030f-4eda-b220-f73c3956a041" (UID: "030d012c-030f-4eda-b220-f73c3956a041"). InnerVolumeSpecName "kube-api-access-lhmqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:03:44.359164 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.359084    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pw788\" (UniqueName: \"kubernetes.io/projected/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-kube-api-access-pw788\") pod \"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f\" (UID: \"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f\") "
Aug 05 18:03:44.359282 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.359187    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-maas-api-tls\") pod \"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f\" (UID: \"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f\") "
Aug 05 18:03:44.359282 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.359245    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-openshift-service-ca\") pod \"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f\" (UID: \"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f\") "
Aug 05 18:03:44.359555 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.359523    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/030d012c-030f-4eda-b220-f73c3956a041-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:03:44.359693 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.359567    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhmqh\" (UniqueName: \"kubernetes.io/projected/030d012c-030f-4eda-b220-f73c3956a041-kube-api-access-lhmqh\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:03:44.359693 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.359575    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5b1d7850-d389-4a17-a99b-e7b2a1e4b85f" (UID: "5b1d7850-d389-4a17-a99b-e7b2a1e4b85f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:03:44.359693 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.359585    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqcx4\" (UniqueName: \"kubernetes.io/projected/7b76ea36-25d6-464a-acb3-379bec538d3e-kube-api-access-sqcx4\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:03:44.359693 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.359619    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b76ea36-25d6-464a-acb3-379bec538d3e-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:03:44.361143 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.361117    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5b1d7850-d389-4a17-a99b-e7b2a1e4b85f" (UID: "5b1d7850-d389-4a17-a99b-e7b2a1e4b85f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:03:44.361143 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.361132    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-kube-api-access-pw788" (OuterVolumeSpecName: "kube-api-access-pw788") pod "5b1d7850-d389-4a17-a99b-e7b2a1e4b85f" (UID: "5b1d7850-d389-4a17-a99b-e7b2a1e4b85f"). InnerVolumeSpecName "kube-api-access-pw788". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:03:44.460388 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.460363    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pw788\" (UniqueName: \"kubernetes.io/projected/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-kube-api-access-pw788\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:03:44.460388 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.460387    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:03:44.460541 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.460397    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:03:44.831832 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.831795    2556 generic.go:356] "Generic (PLEG): container finished" podID="030d012c-030f-4eda-b220-f73c3956a041" containerID="8e4bc6498a125ee1939badd476e4e7f0421a04a11ae34f81c132343acf3de953" exitCode=0
Aug 05 18:03:44.832274 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.831842    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"
Aug 05 18:03:44.832274 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.831882    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn" event={"ID":"030d012c-030f-4eda-b220-f73c3956a041","Type":"ContainerDied","Data":"8e4bc6498a125ee1939badd476e4e7f0421a04a11ae34f81c132343acf3de953"}
Aug 05 18:03:44.832274 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.831935    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn" event={"ID":"030d012c-030f-4eda-b220-f73c3956a041","Type":"ContainerDied","Data":"aae27db6f45996c599681b8b909c3a8449a41bc4aa9d02dec03908838d0bbf9f"}
Aug 05 18:03:44.832274 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.831967    2556 scope.go:117] "RemoveContainer" containerID="8e4bc6498a125ee1939badd476e4e7f0421a04a11ae34f81c132343acf3de953"
Aug 05 18:03:44.833307 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.833285    2556 generic.go:356] "Generic (PLEG): container finished" podID="5b1d7850-d389-4a17-a99b-e7b2a1e4b85f" containerID="86d27cb11574021c71dc9cc3fc264a8e2a8289b22d726d30b562399f523789b6" exitCode=0
Aug 05 18:03:44.833381 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.833331    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"
Aug 05 18:03:44.833438 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.833374    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t" event={"ID":"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f","Type":"ContainerDied","Data":"86d27cb11574021c71dc9cc3fc264a8e2a8289b22d726d30b562399f523789b6"}
Aug 05 18:03:44.833438 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.833419    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t" event={"ID":"5b1d7850-d389-4a17-a99b-e7b2a1e4b85f","Type":"ContainerDied","Data":"abb788f83ac466a1a8b676d0143f392032c8600984e875097b50d38cdf6faa12"}
Aug 05 18:03:44.834875 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.834853    2556 generic.go:356] "Generic (PLEG): container finished" podID="7b76ea36-25d6-464a-acb3-379bec538d3e" containerID="f9d1fa135f3bcc084e86c13b18339142fc5ec3c71e901c4d0ee23f38cc0b0abf" exitCode=0
Aug 05 18:03:44.834983 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.834908    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"
Aug 05 18:03:44.834983 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.834935    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p" event={"ID":"7b76ea36-25d6-464a-acb3-379bec538d3e","Type":"ContainerDied","Data":"f9d1fa135f3bcc084e86c13b18339142fc5ec3c71e901c4d0ee23f38cc0b0abf"}
Aug 05 18:03:44.835073 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.834997    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p" event={"ID":"7b76ea36-25d6-464a-acb3-379bec538d3e","Type":"ContainerDied","Data":"9a4b6f90c224e5903c4f9d96adc26e79bbc7ee3bdcfe270a3c6b8c1a0b818b9c"}
Aug 05 18:03:44.844616 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.844407    2556 scope.go:117] "RemoveContainer" containerID="8e4bc6498a125ee1939badd476e4e7f0421a04a11ae34f81c132343acf3de953"
Aug 05 18:03:44.845350 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:03:44.845322    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e4bc6498a125ee1939badd476e4e7f0421a04a11ae34f81c132343acf3de953\": container with ID starting with 8e4bc6498a125ee1939badd476e4e7f0421a04a11ae34f81c132343acf3de953 not found: ID does not exist" containerID="8e4bc6498a125ee1939badd476e4e7f0421a04a11ae34f81c132343acf3de953"
Aug 05 18:03:44.845456 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.845361    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e4bc6498a125ee1939badd476e4e7f0421a04a11ae34f81c132343acf3de953"} err="failed to get container status \"8e4bc6498a125ee1939badd476e4e7f0421a04a11ae34f81c132343acf3de953\": rpc error: code = NotFound desc = could not find container \"8e4bc6498a125ee1939badd476e4e7f0421a04a11ae34f81c132343acf3de953\": container with ID starting with 8e4bc6498a125ee1939badd476e4e7f0421a04a11ae34f81c132343acf3de953 not found: ID does not exist"
Aug 05 18:03:44.845456 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.845384    2556 scope.go:117] "RemoveContainer" containerID="86d27cb11574021c71dc9cc3fc264a8e2a8289b22d726d30b562399f523789b6"
Aug 05 18:03:44.854241 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.854227    2556 scope.go:117] "RemoveContainer" containerID="86d27cb11574021c71dc9cc3fc264a8e2a8289b22d726d30b562399f523789b6"
Aug 05 18:03:44.854467 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:03:44.854448    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86d27cb11574021c71dc9cc3fc264a8e2a8289b22d726d30b562399f523789b6\": container with ID starting with 86d27cb11574021c71dc9cc3fc264a8e2a8289b22d726d30b562399f523789b6 not found: ID does not exist" containerID="86d27cb11574021c71dc9cc3fc264a8e2a8289b22d726d30b562399f523789b6"
Aug 05 18:03:44.854528 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.854474    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86d27cb11574021c71dc9cc3fc264a8e2a8289b22d726d30b562399f523789b6"} err="failed to get container status \"86d27cb11574021c71dc9cc3fc264a8e2a8289b22d726d30b562399f523789b6\": rpc error: code = NotFound desc = could not find container \"86d27cb11574021c71dc9cc3fc264a8e2a8289b22d726d30b562399f523789b6\": container with ID starting with 86d27cb11574021c71dc9cc3fc264a8e2a8289b22d726d30b562399f523789b6 not found: ID does not exist"
Aug 05 18:03:44.854528 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.854489    2556 scope.go:117] "RemoveContainer" containerID="f9d1fa135f3bcc084e86c13b18339142fc5ec3c71e901c4d0ee23f38cc0b0abf"
Aug 05 18:03:44.860429 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.860409    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"]
Aug 05 18:03:44.863050 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.863028    2556 scope.go:117] "RemoveContainer" containerID="f9d1fa135f3bcc084e86c13b18339142fc5ec3c71e901c4d0ee23f38cc0b0abf"
Aug 05 18:03:44.863315 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:03:44.863296    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9d1fa135f3bcc084e86c13b18339142fc5ec3c71e901c4d0ee23f38cc0b0abf\": container with ID starting with f9d1fa135f3bcc084e86c13b18339142fc5ec3c71e901c4d0ee23f38cc0b0abf not found: ID does not exist" containerID="f9d1fa135f3bcc084e86c13b18339142fc5ec3c71e901c4d0ee23f38cc0b0abf"
Aug 05 18:03:44.863390 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.863326    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9d1fa135f3bcc084e86c13b18339142fc5ec3c71e901c4d0ee23f38cc0b0abf"} err="failed to get container status \"f9d1fa135f3bcc084e86c13b18339142fc5ec3c71e901c4d0ee23f38cc0b0abf\": rpc error: code = NotFound desc = could not find container \"f9d1fa135f3bcc084e86c13b18339142fc5ec3c71e901c4d0ee23f38cc0b0abf\": container with ID starting with f9d1fa135f3bcc084e86c13b18339142fc5ec3c71e901c4d0ee23f38cc0b0abf not found: ID does not exist"
Aug 05 18:03:44.863390 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.863352    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-19f4b3-664757f64d-ktscn"]
Aug 05 18:03:44.872923 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.872902    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"]
Aug 05 18:03:44.876252 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.876234    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-19f4b3-6bc9d95876-lf95p"]
Aug 05 18:03:44.888215 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.888197    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"]
Aug 05 18:03:44.891392 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:44.891375    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-19f4b3-557cfb98-zwt2t"]
Aug 05 18:03:45.097518 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:45.097452    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="030d012c-030f-4eda-b220-f73c3956a041" path="/var/lib/kubelet/pods/030d012c-030f-4eda-b220-f73c3956a041/volumes"
Aug 05 18:03:45.097832 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:45.097819    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b1d7850-d389-4a17-a99b-e7b2a1e4b85f" path="/var/lib/kubelet/pods/5b1d7850-d389-4a17-a99b-e7b2a1e4b85f/volumes"
Aug 05 18:03:45.098134 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:45.098123    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b76ea36-25d6-464a-acb3-379bec538d3e" path="/var/lib/kubelet/pods/7b76ea36-25d6-464a-acb3-379bec538d3e/volumes"
Aug 05 18:03:47.252731 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.252694    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4" podUID="2fefa718-7a57-4888-ba8c-6028e617dc8e" containerName="main" containerID="cri-o://9b54108cb8fbd6f7b5e23b8e1e13bcf37f3954c052a241b47fef279821bd553d" gracePeriod=45
Aug 05 18:03:47.500168 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.500127    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:03:47.685489 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.685402    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-home\") pod \"2fefa718-7a57-4888-ba8c-6028e617dc8e\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") "
Aug 05 18:03:47.685489 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.685438    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-model-cache\") pod \"2fefa718-7a57-4888-ba8c-6028e617dc8e\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") "
Aug 05 18:03:47.685489 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.685464    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-dshm\") pod \"2fefa718-7a57-4888-ba8c-6028e617dc8e\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") "
Aug 05 18:03:47.685489 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.685486    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fefa718-7a57-4888-ba8c-6028e617dc8e-tls-certs\") pod \"2fefa718-7a57-4888-ba8c-6028e617dc8e\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") "
Aug 05 18:03:47.685825 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.685507    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-tmp-dir\") pod \"2fefa718-7a57-4888-ba8c-6028e617dc8e\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") "
Aug 05 18:03:47.685825 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.685592    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hs69\" (UniqueName: \"kubernetes.io/projected/2fefa718-7a57-4888-ba8c-6028e617dc8e-kube-api-access-8hs69\") pod \"2fefa718-7a57-4888-ba8c-6028e617dc8e\" (UID: \"2fefa718-7a57-4888-ba8c-6028e617dc8e\") "
Aug 05 18:03:47.685825 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.685733    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-home" (OuterVolumeSpecName: "home") pod "2fefa718-7a57-4888-ba8c-6028e617dc8e" (UID: "2fefa718-7a57-4888-ba8c-6028e617dc8e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:03:47.685825 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.685753    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-model-cache" (OuterVolumeSpecName: "model-cache") pod "2fefa718-7a57-4888-ba8c-6028e617dc8e" (UID: "2fefa718-7a57-4888-ba8c-6028e617dc8e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:03:47.685825 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.685790    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2fefa718-7a57-4888-ba8c-6028e617dc8e" (UID: "2fefa718-7a57-4888-ba8c-6028e617dc8e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:03:47.686056 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.685943    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-home\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:03:47.686056 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.685962    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-model-cache\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:03:47.686056 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.686001    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-tmp-dir\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:03:47.687734 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.687711    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-dshm" (OuterVolumeSpecName: "dshm") pod "2fefa718-7a57-4888-ba8c-6028e617dc8e" (UID: "2fefa718-7a57-4888-ba8c-6028e617dc8e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:03:47.687836 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.687815    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fefa718-7a57-4888-ba8c-6028e617dc8e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2fefa718-7a57-4888-ba8c-6028e617dc8e" (UID: "2fefa718-7a57-4888-ba8c-6028e617dc8e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:03:47.687885 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.687817    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fefa718-7a57-4888-ba8c-6028e617dc8e-kube-api-access-8hs69" (OuterVolumeSpecName: "kube-api-access-8hs69") pod "2fefa718-7a57-4888-ba8c-6028e617dc8e" (UID: "2fefa718-7a57-4888-ba8c-6028e617dc8e"). InnerVolumeSpecName "kube-api-access-8hs69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:03:47.787239 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.787202    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hs69\" (UniqueName: \"kubernetes.io/projected/2fefa718-7a57-4888-ba8c-6028e617dc8e-kube-api-access-8hs69\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:03:47.787239 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.787234    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fefa718-7a57-4888-ba8c-6028e617dc8e-dshm\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:03:47.787239 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.787246    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fefa718-7a57-4888-ba8c-6028e617dc8e-tls-certs\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:03:47.853500 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.853464    2556 generic.go:356] "Generic (PLEG): container finished" podID="2fefa718-7a57-4888-ba8c-6028e617dc8e" containerID="9b54108cb8fbd6f7b5e23b8e1e13bcf37f3954c052a241b47fef279821bd553d" exitCode=0
Aug 05 18:03:47.853677 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.853512    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"
Aug 05 18:03:47.853677 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.853551    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4" event={"ID":"2fefa718-7a57-4888-ba8c-6028e617dc8e","Type":"ContainerDied","Data":"9b54108cb8fbd6f7b5e23b8e1e13bcf37f3954c052a241b47fef279821bd553d"}
Aug 05 18:03:47.853677 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.853595    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4" event={"ID":"2fefa718-7a57-4888-ba8c-6028e617dc8e","Type":"ContainerDied","Data":"d5712b382ad3e7b4359636c224b35d79550c1f98fd00321159d046b9725b8b88"}
Aug 05 18:03:47.853677 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.853622    2556 scope.go:117] "RemoveContainer" containerID="9b54108cb8fbd6f7b5e23b8e1e13bcf37f3954c052a241b47fef279821bd553d"
Aug 05 18:03:47.862302 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.862287    2556 scope.go:117] "RemoveContainer" containerID="9b54108cb8fbd6f7b5e23b8e1e13bcf37f3954c052a241b47fef279821bd553d"
Aug 05 18:03:47.862540 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:03:47.862521    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b54108cb8fbd6f7b5e23b8e1e13bcf37f3954c052a241b47fef279821bd553d\": container with ID starting with 9b54108cb8fbd6f7b5e23b8e1e13bcf37f3954c052a241b47fef279821bd553d not found: ID does not exist" containerID="9b54108cb8fbd6f7b5e23b8e1e13bcf37f3954c052a241b47fef279821bd553d"
Aug 05 18:03:47.862592 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.862548    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b54108cb8fbd6f7b5e23b8e1e13bcf37f3954c052a241b47fef279821bd553d"} err="failed to get container status \"9b54108cb8fbd6f7b5e23b8e1e13bcf37f3954c052a241b47fef279821bd553d\": rpc error: code = NotFound desc = could not find container \"9b54108cb8fbd6f7b5e23b8e1e13bcf37f3954c052a241b47fef279821bd553d\": container with ID starting with 9b54108cb8fbd6f7b5e23b8e1e13bcf37f3954c052a241b47fef279821bd553d not found: ID does not exist"
Aug 05 18:03:47.878031 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.878006    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"]
Aug 05 18:03:47.881832 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:47.881812    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-19f4b3/test-model-19f4b3-kserve-85f6c68cff-tcpq4"]
Aug 05 18:03:49.097296 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:03:49.097239    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fefa718-7a57-4888-ba8c-6028e617dc8e" path="/var/lib/kubelet/pods/2fefa718-7a57-4888-ba8c-6028e617dc8e/volumes"
Aug 05 18:04:01.263983 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.263950    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"]
Aug 05 18:04:01.264785 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.264731    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq" podUID="58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9" containerName="maas-api" containerID="cri-o://ad44322d21560a79f4d700829c9936c55941bdcdd65e43cff925b0b7cdce3b8b" gracePeriod=30
Aug 05 18:04:01.335132 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.335088    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"]
Aug 05 18:04:01.335930 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.335344    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc" podUID="353f8056-3519-448f-89bc-9ae4bf841932" containerName="payload-pre-processing" containerID="cri-o://783705bf6ae9da146d20989546ec8578337e00196fed9f3f0fbec86f05ac0127" gracePeriod=30
Aug 05 18:04:01.585382 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.585354    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"
Aug 05 18:04:01.598948 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.598920    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-openshift-service-ca\") pod \"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9\" (UID: \"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9\") "
Aug 05 18:04:01.599085 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.598965    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nl76q\" (UniqueName: \"kubernetes.io/projected/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-kube-api-access-nl76q\") pod \"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9\" (UID: \"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9\") "
Aug 05 18:04:01.599085 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.598984    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-maas-api-tls\") pod \"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9\" (UID: \"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9\") "
Aug 05 18:04:01.599314 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.599269    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9" (UID: "58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:04:01.601477 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.601448    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9" (UID: "58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:04:01.601568 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.601506    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-kube-api-access-nl76q" (OuterVolumeSpecName: "kube-api-access-nl76q") pod "58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9" (UID: "58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9"). InnerVolumeSpecName "kube-api-access-nl76q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:04:01.626692 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.626669    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"
Aug 05 18:04:01.699855 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.699821    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwtq8\" (UniqueName: \"kubernetes.io/projected/353f8056-3519-448f-89bc-9ae4bf841932-kube-api-access-fwtq8\") pod \"353f8056-3519-448f-89bc-9ae4bf841932\" (UID: \"353f8056-3519-448f-89bc-9ae4bf841932\") "
Aug 05 18:04:01.700020 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.699870    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/353f8056-3519-448f-89bc-9ae4bf841932-plugins-config-volume\") pod \"353f8056-3519-448f-89bc-9ae4bf841932\" (UID: \"353f8056-3519-448f-89bc-9ae4bf841932\") "
Aug 05 18:04:01.700065 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.700029    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:04:01.700065 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.700040    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nl76q\" (UniqueName: \"kubernetes.io/projected/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-kube-api-access-nl76q\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:04:01.700065 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.700050    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:04:01.700312 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.700281    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/353f8056-3519-448f-89bc-9ae4bf841932-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "353f8056-3519-448f-89bc-9ae4bf841932" (UID: "353f8056-3519-448f-89bc-9ae4bf841932"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:04:01.701834 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.701815    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/353f8056-3519-448f-89bc-9ae4bf841932-kube-api-access-fwtq8" (OuterVolumeSpecName: "kube-api-access-fwtq8") pod "353f8056-3519-448f-89bc-9ae4bf841932" (UID: "353f8056-3519-448f-89bc-9ae4bf841932"). InnerVolumeSpecName "kube-api-access-fwtq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:04:01.800999 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.800939    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwtq8\" (UniqueName: \"kubernetes.io/projected/353f8056-3519-448f-89bc-9ae4bf841932-kube-api-access-fwtq8\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:04:01.800999 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.800958    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/353f8056-3519-448f-89bc-9ae4bf841932-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:04:01.918987 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.918954    2556 generic.go:356] "Generic (PLEG): container finished" podID="58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9" containerID="ad44322d21560a79f4d700829c9936c55941bdcdd65e43cff925b0b7cdce3b8b" exitCode=0
Aug 05 18:04:01.919192 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.919001    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"
Aug 05 18:04:01.919192 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.919040    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq" event={"ID":"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9","Type":"ContainerDied","Data":"ad44322d21560a79f4d700829c9936c55941bdcdd65e43cff925b0b7cdce3b8b"}
Aug 05 18:04:01.919192 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.919093    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq" event={"ID":"58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9","Type":"ContainerDied","Data":"50ec4304626e3fe7aa3622c8a69f4e7897501e30bb0359f0b29cf61583a7dce3"}
Aug 05 18:04:01.919192 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.919123    2556 scope.go:117] "RemoveContainer" containerID="ad44322d21560a79f4d700829c9936c55941bdcdd65e43cff925b0b7cdce3b8b"
Aug 05 18:04:01.920629 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.920593    2556 generic.go:356] "Generic (PLEG): container finished" podID="353f8056-3519-448f-89bc-9ae4bf841932" containerID="783705bf6ae9da146d20989546ec8578337e00196fed9f3f0fbec86f05ac0127" exitCode=0
Aug 05 18:04:01.920727 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.920634    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"
Aug 05 18:04:01.920727 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.920663    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc" event={"ID":"353f8056-3519-448f-89bc-9ae4bf841932","Type":"ContainerDied","Data":"783705bf6ae9da146d20989546ec8578337e00196fed9f3f0fbec86f05ac0127"}
Aug 05 18:04:01.920828 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.920726    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc" event={"ID":"353f8056-3519-448f-89bc-9ae4bf841932","Type":"ContainerDied","Data":"2fd83fcc3f1ee8591f9996e4242335bec4adf3b352fff0cfada05f1d4914c9fd"}
Aug 05 18:04:01.928713 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.928582    2556 scope.go:117] "RemoveContainer" containerID="ad44322d21560a79f4d700829c9936c55941bdcdd65e43cff925b0b7cdce3b8b"
Aug 05 18:04:01.928871 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:04:01.928852    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad44322d21560a79f4d700829c9936c55941bdcdd65e43cff925b0b7cdce3b8b\": container with ID starting with ad44322d21560a79f4d700829c9936c55941bdcdd65e43cff925b0b7cdce3b8b not found: ID does not exist" containerID="ad44322d21560a79f4d700829c9936c55941bdcdd65e43cff925b0b7cdce3b8b"
Aug 05 18:04:01.928935 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.928879    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad44322d21560a79f4d700829c9936c55941bdcdd65e43cff925b0b7cdce3b8b"} err="failed to get container status \"ad44322d21560a79f4d700829c9936c55941bdcdd65e43cff925b0b7cdce3b8b\": rpc error: code = NotFound desc = could not find container \"ad44322d21560a79f4d700829c9936c55941bdcdd65e43cff925b0b7cdce3b8b\": container with ID starting with ad44322d21560a79f4d700829c9936c55941bdcdd65e43cff925b0b7cdce3b8b not found: ID does not exist"
Aug 05 18:04:01.928935 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.928896    2556 scope.go:117] "RemoveContainer" containerID="783705bf6ae9da146d20989546ec8578337e00196fed9f3f0fbec86f05ac0127"
Aug 05 18:04:01.936736 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.936715    2556 scope.go:117] "RemoveContainer" containerID="783705bf6ae9da146d20989546ec8578337e00196fed9f3f0fbec86f05ac0127"
Aug 05 18:04:01.936992 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:04:01.936975    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"783705bf6ae9da146d20989546ec8578337e00196fed9f3f0fbec86f05ac0127\": container with ID starting with 783705bf6ae9da146d20989546ec8578337e00196fed9f3f0fbec86f05ac0127 not found: ID does not exist" containerID="783705bf6ae9da146d20989546ec8578337e00196fed9f3f0fbec86f05ac0127"
Aug 05 18:04:01.937062 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.936998    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"783705bf6ae9da146d20989546ec8578337e00196fed9f3f0fbec86f05ac0127"} err="failed to get container status \"783705bf6ae9da146d20989546ec8578337e00196fed9f3f0fbec86f05ac0127\": rpc error: code = NotFound desc = could not find container \"783705bf6ae9da146d20989546ec8578337e00196fed9f3f0fbec86f05ac0127\": container with ID starting with 783705bf6ae9da146d20989546ec8578337e00196fed9f3f0fbec86f05ac0127 not found: ID does not exist"
Aug 05 18:04:01.943448 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.943427    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"]
Aug 05 18:04:01.946421 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.946399    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-76d454-6889b7d684-g2mbc"]
Aug 05 18:04:01.959716 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.959683    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"]
Aug 05 18:04:01.962116 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:01.962093    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-76d454-84f8db8f74-pvhqq"]
Aug 05 18:04:03.096601 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:03.096566    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="353f8056-3519-448f-89bc-9ae4bf841932" path="/var/lib/kubelet/pods/353f8056-3519-448f-89bc-9ae4bf841932/volumes"
Aug 05 18:04:03.096966 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:03.096941    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9" path="/var/lib/kubelet/pods/58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9/volumes"
Aug 05 18:04:12.815933 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.815893    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"]
Aug 05 18:04:12.816625 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816433    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b1d7850-d389-4a17-a99b-e7b2a1e4b85f" containerName="maas-api"
Aug 05 18:04:12.816625 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816448    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b1d7850-d389-4a17-a99b-e7b2a1e4b85f" containerName="maas-api"
Aug 05 18:04:12.816625 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816464    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="030d012c-030f-4eda-b220-f73c3956a041" containerName="payload-processing"
Aug 05 18:04:12.816625 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816474    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="030d012c-030f-4eda-b220-f73c3956a041" containerName="payload-processing"
Aug 05 18:04:12.816625 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816489    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b76ea36-25d6-464a-acb3-379bec538d3e" containerName="payload-pre-processing"
Aug 05 18:04:12.816625 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816498    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b76ea36-25d6-464a-acb3-379bec538d3e" containerName="payload-pre-processing"
Aug 05 18:04:12.816625 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816512    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9" containerName="maas-api"
Aug 05 18:04:12.816625 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816520    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9" containerName="maas-api"
Aug 05 18:04:12.816625 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816532    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="353f8056-3519-448f-89bc-9ae4bf841932" containerName="payload-pre-processing"
Aug 05 18:04:12.816625 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816541    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="353f8056-3519-448f-89bc-9ae4bf841932" containerName="payload-pre-processing"
Aug 05 18:04:12.816625 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816568    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fefa718-7a57-4888-ba8c-6028e617dc8e" containerName="main"
Aug 05 18:04:12.816625 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816577    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fefa718-7a57-4888-ba8c-6028e617dc8e" containerName="main"
Aug 05 18:04:12.817097 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816656    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fefa718-7a57-4888-ba8c-6028e617dc8e" containerName="main"
Aug 05 18:04:12.817097 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816669    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b76ea36-25d6-464a-acb3-379bec538d3e" containerName="payload-pre-processing"
Aug 05 18:04:12.817097 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816678    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="58f4103f-fc3f-4cc1-b7a8-9a2c0356fdf9" containerName="maas-api"
Aug 05 18:04:12.817097 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816686    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b1d7850-d389-4a17-a99b-e7b2a1e4b85f" containerName="maas-api"
Aug 05 18:04:12.817097 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816695    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="030d012c-030f-4eda-b220-f73c3956a041" containerName="payload-processing"
Aug 05 18:04:12.817097 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.816704    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="353f8056-3519-448f-89bc-9ae4bf841932" containerName="payload-pre-processing"
Aug 05 18:04:12.821212 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.821187    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:04:12.823642 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.823618    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ql6kj\""
Aug 05 18:04:12.823751 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.823621    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-d5bdd1-serving-cert\""
Aug 05 18:04:12.829721 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.829118    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"]
Aug 05 18:04:12.896116 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.896087    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d58336-31a7-49bd-b729-704774c92f1e-maas-api-tls\") pod \"maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484\" (UID: \"04d58336-31a7-49bd-b729-704774c92f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:04:12.896274 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.896126    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwbh6\" (UniqueName: \"kubernetes.io/projected/04d58336-31a7-49bd-b729-704774c92f1e-kube-api-access-fwbh6\") pod \"maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484\" (UID: \"04d58336-31a7-49bd-b729-704774c92f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:04:12.896274 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.896223    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d58336-31a7-49bd-b729-704774c92f1e-openshift-service-ca\") pod \"maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484\" (UID: \"04d58336-31a7-49bd-b729-704774c92f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:04:12.996804 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.996768    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d58336-31a7-49bd-b729-704774c92f1e-openshift-service-ca\") pod \"maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484\" (UID: \"04d58336-31a7-49bd-b729-704774c92f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:04:12.996995 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.996828    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d58336-31a7-49bd-b729-704774c92f1e-maas-api-tls\") pod \"maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484\" (UID: \"04d58336-31a7-49bd-b729-704774c92f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:04:12.996995 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.996854    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwbh6\" (UniqueName: \"kubernetes.io/projected/04d58336-31a7-49bd-b729-704774c92f1e-kube-api-access-fwbh6\") pod \"maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484\" (UID: \"04d58336-31a7-49bd-b729-704774c92f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:04:12.997125 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:04:12.997007    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-serving-cert: secret "maas-api-e2e-api-a-d5bdd1-serving-cert" not found
Aug 05 18:04:12.997125 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:04:12.997073    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04d58336-31a7-49bd-b729-704774c92f1e-maas-api-tls podName:04d58336-31a7-49bd-b729-704774c92f1e nodeName:}" failed. No retries permitted until 2026-08-05 18:04:13.497051507 +0000 UTC m=+2666.915366246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/04d58336-31a7-49bd-b729-704774c92f1e-maas-api-tls") pod "maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484" (UID: "04d58336-31a7-49bd-b729-704774c92f1e") : secret "maas-api-e2e-api-a-d5bdd1-serving-cert" not found
Aug 05 18:04:12.997529 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:12.997506    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d58336-31a7-49bd-b729-704774c92f1e-openshift-service-ca\") pod \"maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484\" (UID: \"04d58336-31a7-49bd-b729-704774c92f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:04:13.007305 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.007283    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwbh6\" (UniqueName: \"kubernetes.io/projected/04d58336-31a7-49bd-b729-704774c92f1e-kube-api-access-fwbh6\") pod \"maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484\" (UID: \"04d58336-31a7-49bd-b729-704774c92f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:04:13.444323 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.444287    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"]
Aug 05 18:04:13.449927 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.449901    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"
Aug 05 18:04:13.455686 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.455661    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-d5bdd1\""
Aug 05 18:04:13.456277 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.456137    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-d5bdd1-dockercfg-ckq9s\""
Aug 05 18:04:13.457589 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.457562    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"]
Aug 05 18:04:13.500268 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.500236    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dcmg\" (UniqueName: \"kubernetes.io/projected/9855ca10-9f8f-4a3b-a0ae-02644a8ba79f-kube-api-access-7dcmg\") pod \"payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl\" (UID: \"9855ca10-9f8f-4a3b-a0ae-02644a8ba79f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"
Aug 05 18:04:13.500402 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.500288    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9855ca10-9f8f-4a3b-a0ae-02644a8ba79f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl\" (UID: \"9855ca10-9f8f-4a3b-a0ae-02644a8ba79f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"
Aug 05 18:04:13.500402 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.500358    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d58336-31a7-49bd-b729-704774c92f1e-maas-api-tls\") pod \"maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484\" (UID: \"04d58336-31a7-49bd-b729-704774c92f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:04:13.502700 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.502675    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d58336-31a7-49bd-b729-704774c92f1e-maas-api-tls\") pod \"maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484\" (UID: \"04d58336-31a7-49bd-b729-704774c92f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:04:13.601610 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.601573    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9855ca10-9f8f-4a3b-a0ae-02644a8ba79f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl\" (UID: \"9855ca10-9f8f-4a3b-a0ae-02644a8ba79f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"
Aug 05 18:04:13.601774 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.601659    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dcmg\" (UniqueName: \"kubernetes.io/projected/9855ca10-9f8f-4a3b-a0ae-02644a8ba79f-kube-api-access-7dcmg\") pod \"payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl\" (UID: \"9855ca10-9f8f-4a3b-a0ae-02644a8ba79f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"
Aug 05 18:04:13.602281 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.602261    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9855ca10-9f8f-4a3b-a0ae-02644a8ba79f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl\" (UID: \"9855ca10-9f8f-4a3b-a0ae-02644a8ba79f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"
Aug 05 18:04:13.611442 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.611414    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dcmg\" (UniqueName: \"kubernetes.io/projected/9855ca10-9f8f-4a3b-a0ae-02644a8ba79f-kube-api-access-7dcmg\") pod \"payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl\" (UID: \"9855ca10-9f8f-4a3b-a0ae-02644a8ba79f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"
Aug 05 18:04:13.733552 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.733520    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:04:13.762619 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.762595    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"
Aug 05 18:04:13.881576 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.881544    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"]
Aug 05 18:04:13.886756 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.886733    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 18:04:13.906446 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.906391    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"]
Aug 05 18:04:13.909050 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:04:13.908978    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9855ca10_9f8f_4a3b_a0ae_02644a8ba79f.slice/crio-6bf4f2b35b721b3f5c6d2559fb5cae6b1c2a6d69c333dfb90c29dde02bd833fe WatchSource:0}: Error finding container 6bf4f2b35b721b3f5c6d2559fb5cae6b1c2a6d69c333dfb90c29dde02bd833fe: Status 404 returned error can't find the container with id 6bf4f2b35b721b3f5c6d2559fb5cae6b1c2a6d69c333dfb90c29dde02bd833fe
Aug 05 18:04:13.980672 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.980643    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484" event={"ID":"04d58336-31a7-49bd-b729-704774c92f1e","Type":"ContainerStarted","Data":"c1a9012db2b214ed4385ee64339c42711548c00f9efb171964367f5dddf72ce4"}
Aug 05 18:04:13.981939 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.981916    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl" event={"ID":"9855ca10-9f8f-4a3b-a0ae-02644a8ba79f","Type":"ContainerStarted","Data":"ee517150c3f0b23c6a1dae3fa48f6289faa425b07e57ed63404eadd3ae2d8137"}
Aug 05 18:04:13.982047 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.981955    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl" event={"ID":"9855ca10-9f8f-4a3b-a0ae-02644a8ba79f","Type":"ContainerStarted","Data":"6bf4f2b35b721b3f5c6d2559fb5cae6b1c2a6d69c333dfb90c29dde02bd833fe"}
Aug 05 18:04:13.982047 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.982035    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"
Aug 05 18:04:13.996939 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:13.996871    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl" podStartSLOduration=0.99685965 podStartE2EDuration="996.85965ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:04:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:04:13.995622974 +0000 UTC m=+2667.413937756" watchObservedRunningTime="2026-08-05 18:04:13.99685965 +0000 UTC m=+2667.415174406"
Aug 05 18:04:14.988164 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:14.988122    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484" event={"ID":"04d58336-31a7-49bd-b729-704774c92f1e","Type":"ContainerStarted","Data":"467b298cd4aca4f6565d24ddfb095b2907f924959cd513de6541ee0b51450b9b"}
Aug 05 18:04:15.006075 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:15.006022    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484" podStartSLOduration=2.679072448 podStartE2EDuration="3.006007129s" totalImagesPullingTime="326.934681ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:04:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:04:15.003976208 +0000 UTC m=+2668.422290981" watchObservedRunningTime="2026-08-05 18:04:15.006007129 +0000 UTC m=+2668.424321886"
Aug 05 18:04:15.992600 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:15.992570    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:04:22.005196 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:22.005164    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:04:24.989898 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:24.989854    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl" podUID="9855ca10-9f8f-4a3b-a0ae-02644a8ba79f" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.70:9004: connect: connection refused"
Aug 05 18:04:34.988683 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:34.988650    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"
Aug 05 18:04:42.545665 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.545587    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"]
Aug 05 18:04:42.548763 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.548747    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"
Aug 05 18:04:42.551425 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.551400    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-adc09f\""
Aug 05 18:04:42.551565 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.551518    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-adc09f-dockercfg-ps62j\""
Aug 05 18:04:42.561671 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.561649    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"]
Aug 05 18:04:42.582447 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.582414    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"]
Aug 05 18:04:42.586308 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.586290    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"
Aug 05 18:04:42.595103 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.595077    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"]
Aug 05 18:04:42.742352 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.742293    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c4l6\" (UniqueName: \"kubernetes.io/projected/09bbdad7-7030-4e66-94a4-dd2a8abbbea4-kube-api-access-9c4l6\") pod \"payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9\" (UID: \"09bbdad7-7030-4e66-94a4-dd2a8abbbea4\") " pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"
Aug 05 18:04:42.742352 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.742331    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87b6df52-cb87-4380-a539-ece75e93c623-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj\" (UID: \"87b6df52-cb87-4380-a539-ece75e93c623\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"
Aug 05 18:04:42.742576 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.742373    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09bbdad7-7030-4e66-94a4-dd2a8abbbea4-plugins-config-volume\") pod \"payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9\" (UID: \"09bbdad7-7030-4e66-94a4-dd2a8abbbea4\") " pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"
Aug 05 18:04:42.742576 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.742442    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smmkv\" (UniqueName: \"kubernetes.io/projected/87b6df52-cb87-4380-a539-ece75e93c623-kube-api-access-smmkv\") pod \"payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj\" (UID: \"87b6df52-cb87-4380-a539-ece75e93c623\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"
Aug 05 18:04:42.843360 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.843271    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c4l6\" (UniqueName: \"kubernetes.io/projected/09bbdad7-7030-4e66-94a4-dd2a8abbbea4-kube-api-access-9c4l6\") pod \"payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9\" (UID: \"09bbdad7-7030-4e66-94a4-dd2a8abbbea4\") " pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"
Aug 05 18:04:42.843360 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.843321    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87b6df52-cb87-4380-a539-ece75e93c623-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj\" (UID: \"87b6df52-cb87-4380-a539-ece75e93c623\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"
Aug 05 18:04:42.843360 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.843354    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09bbdad7-7030-4e66-94a4-dd2a8abbbea4-plugins-config-volume\") pod \"payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9\" (UID: \"09bbdad7-7030-4e66-94a4-dd2a8abbbea4\") " pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"
Aug 05 18:04:42.843603 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.843388    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smmkv\" (UniqueName: \"kubernetes.io/projected/87b6df52-cb87-4380-a539-ece75e93c623-kube-api-access-smmkv\") pod \"payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj\" (UID: \"87b6df52-cb87-4380-a539-ece75e93c623\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"
Aug 05 18:04:42.844134 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.844115    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09bbdad7-7030-4e66-94a4-dd2a8abbbea4-plugins-config-volume\") pod \"payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9\" (UID: \"09bbdad7-7030-4e66-94a4-dd2a8abbbea4\") " pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"
Aug 05 18:04:42.844204 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.844117    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87b6df52-cb87-4380-a539-ece75e93c623-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj\" (UID: \"87b6df52-cb87-4380-a539-ece75e93c623\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"
Aug 05 18:04:42.850988 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.850964    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smmkv\" (UniqueName: \"kubernetes.io/projected/87b6df52-cb87-4380-a539-ece75e93c623-kube-api-access-smmkv\") pod \"payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj\" (UID: \"87b6df52-cb87-4380-a539-ece75e93c623\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"
Aug 05 18:04:42.851094 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.850994    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c4l6\" (UniqueName: \"kubernetes.io/projected/09bbdad7-7030-4e66-94a4-dd2a8abbbea4-kube-api-access-9c4l6\") pod \"payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9\" (UID: \"09bbdad7-7030-4e66-94a4-dd2a8abbbea4\") " pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"
Aug 05 18:04:42.858840 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.858820    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"
Aug 05 18:04:42.898059 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:42.898026    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"
Aug 05 18:04:43.012248 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:43.008965    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"]
Aug 05 18:04:43.054444 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:43.054419    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"]
Aug 05 18:04:43.055291 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:04:43.055261    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09bbdad7_7030_4e66_94a4_dd2a8abbbea4.slice/crio-5fcd9a0281da0402694eee6abfba6136d6db383dc1b155d4ec6f6714c2775e31 WatchSource:0}: Error finding container 5fcd9a0281da0402694eee6abfba6136d6db383dc1b155d4ec6f6714c2775e31: Status 404 returned error can't find the container with id 5fcd9a0281da0402694eee6abfba6136d6db383dc1b155d4ec6f6714c2775e31
Aug 05 18:04:43.125932 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:43.125898    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj" event={"ID":"87b6df52-cb87-4380-a539-ece75e93c623","Type":"ContainerStarted","Data":"44aae1b60922ad6c3eadee3adeab606775d121582cf386b4244288bbc662e025"}
Aug 05 18:04:43.126070 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:43.125965    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"
Aug 05 18:04:43.126070 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:43.125995    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj" event={"ID":"87b6df52-cb87-4380-a539-ece75e93c623","Type":"ContainerStarted","Data":"da958a4fbc5b1ded07f84dc9bcbd103798023755cfbf00353571192b7cc719a9"}
Aug 05 18:04:43.127700 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:43.127671    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9" event={"ID":"09bbdad7-7030-4e66-94a4-dd2a8abbbea4","Type":"ContainerStarted","Data":"6628b18498d8b4a35076abf9a15a183c5b0e3303a3b3e7e552a29fd27db8ee0b"}
Aug 05 18:04:43.127815 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:43.127717    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9" event={"ID":"09bbdad7-7030-4e66-94a4-dd2a8abbbea4","Type":"ContainerStarted","Data":"5fcd9a0281da0402694eee6abfba6136d6db383dc1b155d4ec6f6714c2775e31"}
Aug 05 18:04:43.127815 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:43.127756    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"
Aug 05 18:04:43.142392 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:43.142354    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj" podStartSLOduration=1.142339555 podStartE2EDuration="1.142339555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:04:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:04:43.140339244 +0000 UTC m=+2696.558654024" watchObservedRunningTime="2026-08-05 18:04:43.142339555 +0000 UTC m=+2696.560654363"
Aug 05 18:04:43.157103 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:43.157065    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9" podStartSLOduration=1.157050446 podStartE2EDuration="1.157050446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:04:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:04:43.154974093 +0000 UTC m=+2696.573288871" watchObservedRunningTime="2026-08-05 18:04:43.157050446 +0000 UTC m=+2696.575365202"
Aug 05 18:04:54.135322 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:54.135292    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"
Aug 05 18:04:54.135761 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:04:54.135492    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"
Aug 05 18:05:24.529928 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.529891    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"]
Aug 05 18:05:24.530487 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.530132    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484" podUID="04d58336-31a7-49bd-b729-704774c92f1e" containerName="maas-api" containerID="cri-o://467b298cd4aca4f6565d24ddfb095b2907f924959cd513de6541ee0b51450b9b" gracePeriod=30
Aug 05 18:05:24.578558 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.578526    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"]
Aug 05 18:05:24.578811 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.578755    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl" podUID="9855ca10-9f8f-4a3b-a0ae-02644a8ba79f" containerName="payload-pre-processing" containerID="cri-o://ee517150c3f0b23c6a1dae3fa48f6289faa425b07e57ed63404eadd3ae2d8137" gracePeriod=30
Aug 05 18:05:24.838909 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.838882    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:05:24.872849 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.872824    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"
Aug 05 18:05:24.962744 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.962720    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwbh6\" (UniqueName: \"kubernetes.io/projected/04d58336-31a7-49bd-b729-704774c92f1e-kube-api-access-fwbh6\") pod \"04d58336-31a7-49bd-b729-704774c92f1e\" (UID: \"04d58336-31a7-49bd-b729-704774c92f1e\") "
Aug 05 18:05:24.962903 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.962802    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d58336-31a7-49bd-b729-704774c92f1e-openshift-service-ca\") pod \"04d58336-31a7-49bd-b729-704774c92f1e\" (UID: \"04d58336-31a7-49bd-b729-704774c92f1e\") "
Aug 05 18:05:24.962903 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.962826    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d58336-31a7-49bd-b729-704774c92f1e-maas-api-tls\") pod \"04d58336-31a7-49bd-b729-704774c92f1e\" (UID: \"04d58336-31a7-49bd-b729-704774c92f1e\") "
Aug 05 18:05:24.962903 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.962847    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dcmg\" (UniqueName: \"kubernetes.io/projected/9855ca10-9f8f-4a3b-a0ae-02644a8ba79f-kube-api-access-7dcmg\") pod \"9855ca10-9f8f-4a3b-a0ae-02644a8ba79f\" (UID: \"9855ca10-9f8f-4a3b-a0ae-02644a8ba79f\") "
Aug 05 18:05:24.962903 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.962869    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9855ca10-9f8f-4a3b-a0ae-02644a8ba79f-plugins-config-volume\") pod \"9855ca10-9f8f-4a3b-a0ae-02644a8ba79f\" (UID: \"9855ca10-9f8f-4a3b-a0ae-02644a8ba79f\") "
Aug 05 18:05:24.963320 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.963285    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04d58336-31a7-49bd-b729-704774c92f1e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "04d58336-31a7-49bd-b729-704774c92f1e" (UID: "04d58336-31a7-49bd-b729-704774c92f1e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:05:24.963433 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.963354    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9855ca10-9f8f-4a3b-a0ae-02644a8ba79f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9855ca10-9f8f-4a3b-a0ae-02644a8ba79f" (UID: "9855ca10-9f8f-4a3b-a0ae-02644a8ba79f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:05:24.964802 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.964770    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04d58336-31a7-49bd-b729-704774c92f1e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "04d58336-31a7-49bd-b729-704774c92f1e" (UID: "04d58336-31a7-49bd-b729-704774c92f1e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:05:24.964890 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.964869    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9855ca10-9f8f-4a3b-a0ae-02644a8ba79f-kube-api-access-7dcmg" (OuterVolumeSpecName: "kube-api-access-7dcmg") pod "9855ca10-9f8f-4a3b-a0ae-02644a8ba79f" (UID: "9855ca10-9f8f-4a3b-a0ae-02644a8ba79f"). InnerVolumeSpecName "kube-api-access-7dcmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:05:24.964933 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:24.964898    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04d58336-31a7-49bd-b729-704774c92f1e-kube-api-access-fwbh6" (OuterVolumeSpecName: "kube-api-access-fwbh6") pod "04d58336-31a7-49bd-b729-704774c92f1e" (UID: "04d58336-31a7-49bd-b729-704774c92f1e"). InnerVolumeSpecName "kube-api-access-fwbh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:05:25.063763 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.063702    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d58336-31a7-49bd-b729-704774c92f1e-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:05:25.063763 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.063726    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d58336-31a7-49bd-b729-704774c92f1e-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:05:25.063763 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.063735    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dcmg\" (UniqueName: \"kubernetes.io/projected/9855ca10-9f8f-4a3b-a0ae-02644a8ba79f-kube-api-access-7dcmg\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:05:25.063763 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.063744    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9855ca10-9f8f-4a3b-a0ae-02644a8ba79f-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:05:25.063763 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.063752    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwbh6\" (UniqueName: \"kubernetes.io/projected/04d58336-31a7-49bd-b729-704774c92f1e-kube-api-access-fwbh6\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:05:25.323386 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.323306    2556 generic.go:356] "Generic (PLEG): container finished" podID="04d58336-31a7-49bd-b729-704774c92f1e" containerID="467b298cd4aca4f6565d24ddfb095b2907f924959cd513de6541ee0b51450b9b" exitCode=0
Aug 05 18:05:25.323386 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.323365    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"
Aug 05 18:05:25.323584 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.323398    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484" event={"ID":"04d58336-31a7-49bd-b729-704774c92f1e","Type":"ContainerDied","Data":"467b298cd4aca4f6565d24ddfb095b2907f924959cd513de6541ee0b51450b9b"}
Aug 05 18:05:25.323584 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.323452    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484" event={"ID":"04d58336-31a7-49bd-b729-704774c92f1e","Type":"ContainerDied","Data":"c1a9012db2b214ed4385ee64339c42711548c00f9efb171964367f5dddf72ce4"}
Aug 05 18:05:25.323584 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.323483    2556 scope.go:117] "RemoveContainer" containerID="467b298cd4aca4f6565d24ddfb095b2907f924959cd513de6541ee0b51450b9b"
Aug 05 18:05:25.324758 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.324734    2556 generic.go:356] "Generic (PLEG): container finished" podID="9855ca10-9f8f-4a3b-a0ae-02644a8ba79f" containerID="ee517150c3f0b23c6a1dae3fa48f6289faa425b07e57ed63404eadd3ae2d8137" exitCode=0
Aug 05 18:05:25.324870 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.324768    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"
Aug 05 18:05:25.324870 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.324771    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl" event={"ID":"9855ca10-9f8f-4a3b-a0ae-02644a8ba79f","Type":"ContainerDied","Data":"ee517150c3f0b23c6a1dae3fa48f6289faa425b07e57ed63404eadd3ae2d8137"}
Aug 05 18:05:25.324870 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.324809    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl" event={"ID":"9855ca10-9f8f-4a3b-a0ae-02644a8ba79f","Type":"ContainerDied","Data":"6bf4f2b35b721b3f5c6d2559fb5cae6b1c2a6d69c333dfb90c29dde02bd833fe"}
Aug 05 18:05:25.332454 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.332437    2556 scope.go:117] "RemoveContainer" containerID="467b298cd4aca4f6565d24ddfb095b2907f924959cd513de6541ee0b51450b9b"
Aug 05 18:05:25.332704 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:05:25.332684    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"467b298cd4aca4f6565d24ddfb095b2907f924959cd513de6541ee0b51450b9b\": container with ID starting with 467b298cd4aca4f6565d24ddfb095b2907f924959cd513de6541ee0b51450b9b not found: ID does not exist" containerID="467b298cd4aca4f6565d24ddfb095b2907f924959cd513de6541ee0b51450b9b"
Aug 05 18:05:25.332755 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.332715    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"467b298cd4aca4f6565d24ddfb095b2907f924959cd513de6541ee0b51450b9b"} err="failed to get container status \"467b298cd4aca4f6565d24ddfb095b2907f924959cd513de6541ee0b51450b9b\": rpc error: code = NotFound desc = could not find container \"467b298cd4aca4f6565d24ddfb095b2907f924959cd513de6541ee0b51450b9b\": container with ID starting with 467b298cd4aca4f6565d24ddfb095b2907f924959cd513de6541ee0b51450b9b not found: ID does not exist"
Aug 05 18:05:25.332755 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.332735    2556 scope.go:117] "RemoveContainer" containerID="ee517150c3f0b23c6a1dae3fa48f6289faa425b07e57ed63404eadd3ae2d8137"
Aug 05 18:05:25.340191 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.340167    2556 scope.go:117] "RemoveContainer" containerID="ee517150c3f0b23c6a1dae3fa48f6289faa425b07e57ed63404eadd3ae2d8137"
Aug 05 18:05:25.340477 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:05:25.340460    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee517150c3f0b23c6a1dae3fa48f6289faa425b07e57ed63404eadd3ae2d8137\": container with ID starting with ee517150c3f0b23c6a1dae3fa48f6289faa425b07e57ed63404eadd3ae2d8137 not found: ID does not exist" containerID="ee517150c3f0b23c6a1dae3fa48f6289faa425b07e57ed63404eadd3ae2d8137"
Aug 05 18:05:25.340536 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.340482    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee517150c3f0b23c6a1dae3fa48f6289faa425b07e57ed63404eadd3ae2d8137"} err="failed to get container status \"ee517150c3f0b23c6a1dae3fa48f6289faa425b07e57ed63404eadd3ae2d8137\": rpc error: code = NotFound desc = could not find container \"ee517150c3f0b23c6a1dae3fa48f6289faa425b07e57ed63404eadd3ae2d8137\": container with ID starting with ee517150c3f0b23c6a1dae3fa48f6289faa425b07e57ed63404eadd3ae2d8137 not found: ID does not exist"
Aug 05 18:05:25.342104 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.342085    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"]
Aug 05 18:05:25.345120 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.345101    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d5bdd1-b4f44d6-g6484"]
Aug 05 18:05:25.354676 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.354651    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"]
Aug 05 18:05:25.356344 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:25.356325    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-d5bdd1-999f99c56-89sxl"]
Aug 05 18:05:27.097135 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:27.097100    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04d58336-31a7-49bd-b729-704774c92f1e" path="/var/lib/kubelet/pods/04d58336-31a7-49bd-b729-704774c92f1e/volumes"
Aug 05 18:05:27.097522 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:27.097472    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9855ca10-9f8f-4a3b-a0ae-02644a8ba79f" path="/var/lib/kubelet/pods/9855ca10-9f8f-4a3b-a0ae-02644a8ba79f/volumes"
Aug 05 18:05:46.815694 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:46.815658    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"]
Aug 05 18:05:46.816177 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:46.815898    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9" podUID="09bbdad7-7030-4e66-94a4-dd2a8abbbea4" containerName="payload-processing" containerID="cri-o://6628b18498d8b4a35076abf9a15a183c5b0e3303a3b3e7e552a29fd27db8ee0b" gracePeriod=30
Aug 05 18:05:46.818333 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:46.818305    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"]
Aug 05 18:05:46.818566 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:46.818519    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj" podUID="87b6df52-cb87-4380-a539-ece75e93c623" containerName="payload-pre-processing" containerID="cri-o://44aae1b60922ad6c3eadee3adeab606775d121582cf386b4244288bbc662e025" gracePeriod=30
Aug 05 18:05:47.094793 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.094768    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"
Aug 05 18:05:47.097923 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.097903    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"
Aug 05 18:05:47.244397 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.244367    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09bbdad7-7030-4e66-94a4-dd2a8abbbea4-plugins-config-volume\") pod \"09bbdad7-7030-4e66-94a4-dd2a8abbbea4\" (UID: \"09bbdad7-7030-4e66-94a4-dd2a8abbbea4\") "
Aug 05 18:05:47.244541 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.244408    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9c4l6\" (UniqueName: \"kubernetes.io/projected/09bbdad7-7030-4e66-94a4-dd2a8abbbea4-kube-api-access-9c4l6\") pod \"09bbdad7-7030-4e66-94a4-dd2a8abbbea4\" (UID: \"09bbdad7-7030-4e66-94a4-dd2a8abbbea4\") "
Aug 05 18:05:47.244541 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.244433    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87b6df52-cb87-4380-a539-ece75e93c623-plugins-config-volume\") pod \"87b6df52-cb87-4380-a539-ece75e93c623\" (UID: \"87b6df52-cb87-4380-a539-ece75e93c623\") "
Aug 05 18:05:47.244541 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.244495    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smmkv\" (UniqueName: \"kubernetes.io/projected/87b6df52-cb87-4380-a539-ece75e93c623-kube-api-access-smmkv\") pod \"87b6df52-cb87-4380-a539-ece75e93c623\" (UID: \"87b6df52-cb87-4380-a539-ece75e93c623\") "
Aug 05 18:05:47.244788 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.244764    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09bbdad7-7030-4e66-94a4-dd2a8abbbea4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "09bbdad7-7030-4e66-94a4-dd2a8abbbea4" (UID: "09bbdad7-7030-4e66-94a4-dd2a8abbbea4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:05:47.244871 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.244835    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87b6df52-cb87-4380-a539-ece75e93c623-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "87b6df52-cb87-4380-a539-ece75e93c623" (UID: "87b6df52-cb87-4380-a539-ece75e93c623"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:05:47.246428 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.246406    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09bbdad7-7030-4e66-94a4-dd2a8abbbea4-kube-api-access-9c4l6" (OuterVolumeSpecName: "kube-api-access-9c4l6") pod "09bbdad7-7030-4e66-94a4-dd2a8abbbea4" (UID: "09bbdad7-7030-4e66-94a4-dd2a8abbbea4"). InnerVolumeSpecName "kube-api-access-9c4l6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:05:47.246550 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.246525    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87b6df52-cb87-4380-a539-ece75e93c623-kube-api-access-smmkv" (OuterVolumeSpecName: "kube-api-access-smmkv") pod "87b6df52-cb87-4380-a539-ece75e93c623" (UID: "87b6df52-cb87-4380-a539-ece75e93c623"). InnerVolumeSpecName "kube-api-access-smmkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:05:47.345723 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.345662    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smmkv\" (UniqueName: \"kubernetes.io/projected/87b6df52-cb87-4380-a539-ece75e93c623-kube-api-access-smmkv\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:05:47.345723 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.345684    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09bbdad7-7030-4e66-94a4-dd2a8abbbea4-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:05:47.345723 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.345695    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c4l6\" (UniqueName: \"kubernetes.io/projected/09bbdad7-7030-4e66-94a4-dd2a8abbbea4-kube-api-access-9c4l6\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:05:47.345723 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.345704    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87b6df52-cb87-4380-a539-ece75e93c623-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:05:47.429354 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.429317    2556 generic.go:356] "Generic (PLEG): container finished" podID="09bbdad7-7030-4e66-94a4-dd2a8abbbea4" containerID="6628b18498d8b4a35076abf9a15a183c5b0e3303a3b3e7e552a29fd27db8ee0b" exitCode=0
Aug 05 18:05:47.429461 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.429382    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"
Aug 05 18:05:47.429461 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.429409    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9" event={"ID":"09bbdad7-7030-4e66-94a4-dd2a8abbbea4","Type":"ContainerDied","Data":"6628b18498d8b4a35076abf9a15a183c5b0e3303a3b3e7e552a29fd27db8ee0b"}
Aug 05 18:05:47.429580 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.429506    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9" event={"ID":"09bbdad7-7030-4e66-94a4-dd2a8abbbea4","Type":"ContainerDied","Data":"5fcd9a0281da0402694eee6abfba6136d6db383dc1b155d4ec6f6714c2775e31"}
Aug 05 18:05:47.429580 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.429544    2556 scope.go:117] "RemoveContainer" containerID="6628b18498d8b4a35076abf9a15a183c5b0e3303a3b3e7e552a29fd27db8ee0b"
Aug 05 18:05:47.430760 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.430738    2556 generic.go:356] "Generic (PLEG): container finished" podID="87b6df52-cb87-4380-a539-ece75e93c623" containerID="44aae1b60922ad6c3eadee3adeab606775d121582cf386b4244288bbc662e025" exitCode=0
Aug 05 18:05:47.430878 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.430789    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj" event={"ID":"87b6df52-cb87-4380-a539-ece75e93c623","Type":"ContainerDied","Data":"44aae1b60922ad6c3eadee3adeab606775d121582cf386b4244288bbc662e025"}
Aug 05 18:05:47.430878 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.430830    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj" event={"ID":"87b6df52-cb87-4380-a539-ece75e93c623","Type":"ContainerDied","Data":"da958a4fbc5b1ded07f84dc9bcbd103798023755cfbf00353571192b7cc719a9"}
Aug 05 18:05:47.430878 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.430794    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"
Aug 05 18:05:47.438202 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.438179    2556 scope.go:117] "RemoveContainer" containerID="6628b18498d8b4a35076abf9a15a183c5b0e3303a3b3e7e552a29fd27db8ee0b"
Aug 05 18:05:47.438446 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:05:47.438424    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6628b18498d8b4a35076abf9a15a183c5b0e3303a3b3e7e552a29fd27db8ee0b\": container with ID starting with 6628b18498d8b4a35076abf9a15a183c5b0e3303a3b3e7e552a29fd27db8ee0b not found: ID does not exist" containerID="6628b18498d8b4a35076abf9a15a183c5b0e3303a3b3e7e552a29fd27db8ee0b"
Aug 05 18:05:47.438542 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.438454    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6628b18498d8b4a35076abf9a15a183c5b0e3303a3b3e7e552a29fd27db8ee0b"} err="failed to get container status \"6628b18498d8b4a35076abf9a15a183c5b0e3303a3b3e7e552a29fd27db8ee0b\": rpc error: code = NotFound desc = could not find container \"6628b18498d8b4a35076abf9a15a183c5b0e3303a3b3e7e552a29fd27db8ee0b\": container with ID starting with 6628b18498d8b4a35076abf9a15a183c5b0e3303a3b3e7e552a29fd27db8ee0b not found: ID does not exist"
Aug 05 18:05:47.438542 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.438477    2556 scope.go:117] "RemoveContainer" containerID="44aae1b60922ad6c3eadee3adeab606775d121582cf386b4244288bbc662e025"
Aug 05 18:05:47.446402 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.446386    2556 scope.go:117] "RemoveContainer" containerID="44aae1b60922ad6c3eadee3adeab606775d121582cf386b4244288bbc662e025"
Aug 05 18:05:47.446626 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:05:47.446608    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44aae1b60922ad6c3eadee3adeab606775d121582cf386b4244288bbc662e025\": container with ID starting with 44aae1b60922ad6c3eadee3adeab606775d121582cf386b4244288bbc662e025 not found: ID does not exist" containerID="44aae1b60922ad6c3eadee3adeab606775d121582cf386b4244288bbc662e025"
Aug 05 18:05:47.446692 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.446634    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44aae1b60922ad6c3eadee3adeab606775d121582cf386b4244288bbc662e025"} err="failed to get container status \"44aae1b60922ad6c3eadee3adeab606775d121582cf386b4244288bbc662e025\": rpc error: code = NotFound desc = could not find container \"44aae1b60922ad6c3eadee3adeab606775d121582cf386b4244288bbc662e025\": container with ID starting with 44aae1b60922ad6c3eadee3adeab606775d121582cf386b4244288bbc662e025 not found: ID does not exist"
Aug 05 18:05:47.453494 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.453473    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"]
Aug 05 18:05:47.457865 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.457842    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-adc09f-7c77555948-5r5sj"]
Aug 05 18:05:47.469415 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.469396    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"]
Aug 05 18:05:47.474405 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:47.474385    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-adc09f-cc748fc8d-jbzl9"]
Aug 05 18:05:49.096448 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:49.096407    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09bbdad7-7030-4e66-94a4-dd2a8abbbea4" path="/var/lib/kubelet/pods/09bbdad7-7030-4e66-94a4-dd2a8abbbea4/volumes"
Aug 05 18:05:49.096960 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:49.096938    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87b6df52-cb87-4380-a539-ece75e93c623" path="/var/lib/kubelet/pods/87b6df52-cb87-4380-a539-ece75e93c623/volumes"
Aug 05 18:05:58.374319 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.374286    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"]
Aug 05 18:05:58.374911 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.374621    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9855ca10-9f8f-4a3b-a0ae-02644a8ba79f" containerName="payload-pre-processing"
Aug 05 18:05:58.374911 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.374632    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9855ca10-9f8f-4a3b-a0ae-02644a8ba79f" containerName="payload-pre-processing"
Aug 05 18:05:58.374911 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.374642    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04d58336-31a7-49bd-b729-704774c92f1e" containerName="maas-api"
Aug 05 18:05:58.374911 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.374648    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="04d58336-31a7-49bd-b729-704774c92f1e" containerName="maas-api"
Aug 05 18:05:58.374911 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.374668    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87b6df52-cb87-4380-a539-ece75e93c623" containerName="payload-pre-processing"
Aug 05 18:05:58.374911 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.374674    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="87b6df52-cb87-4380-a539-ece75e93c623" containerName="payload-pre-processing"
Aug 05 18:05:58.374911 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.374683    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09bbdad7-7030-4e66-94a4-dd2a8abbbea4" containerName="payload-processing"
Aug 05 18:05:58.374911 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.374688    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="09bbdad7-7030-4e66-94a4-dd2a8abbbea4" containerName="payload-processing"
Aug 05 18:05:58.374911 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.374740    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="87b6df52-cb87-4380-a539-ece75e93c623" containerName="payload-pre-processing"
Aug 05 18:05:58.374911 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.374749    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="09bbdad7-7030-4e66-94a4-dd2a8abbbea4" containerName="payload-processing"
Aug 05 18:05:58.374911 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.374759    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9855ca10-9f8f-4a3b-a0ae-02644a8ba79f" containerName="payload-pre-processing"
Aug 05 18:05:58.374911 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.374767    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="04d58336-31a7-49bd-b729-704774c92f1e" containerName="maas-api"
Aug 05 18:05:58.377354 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.377333    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"
Aug 05 18:05:58.379973 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.379952    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-3f8e0d-serving-cert\""
Aug 05 18:05:58.380089 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.379953    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ql6kj\""
Aug 05 18:05:58.385969 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.385913    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"]
Aug 05 18:05:58.542227 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.542196    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b43900b-d530-44a1-8e77-d6b412fe2f65-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b\" (UID: \"7b43900b-d530-44a1-8e77-d6b412fe2f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"
Aug 05 18:05:58.542416 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.542268    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b43900b-d530-44a1-8e77-d6b412fe2f65-maas-api-tls\") pod \"maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b\" (UID: \"7b43900b-d530-44a1-8e77-d6b412fe2f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"
Aug 05 18:05:58.542416 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.542340    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w257j\" (UniqueName: \"kubernetes.io/projected/7b43900b-d530-44a1-8e77-d6b412fe2f65-kube-api-access-w257j\") pod \"maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b\" (UID: \"7b43900b-d530-44a1-8e77-d6b412fe2f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"
Aug 05 18:05:58.642942 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.642872    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b43900b-d530-44a1-8e77-d6b412fe2f65-maas-api-tls\") pod \"maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b\" (UID: \"7b43900b-d530-44a1-8e77-d6b412fe2f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"
Aug 05 18:05:58.642942 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.642921    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w257j\" (UniqueName: \"kubernetes.io/projected/7b43900b-d530-44a1-8e77-d6b412fe2f65-kube-api-access-w257j\") pod \"maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b\" (UID: \"7b43900b-d530-44a1-8e77-d6b412fe2f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"
Aug 05 18:05:58.643104 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.642974    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b43900b-d530-44a1-8e77-d6b412fe2f65-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b\" (UID: \"7b43900b-d530-44a1-8e77-d6b412fe2f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"
Aug 05 18:05:58.643728 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.643704    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b43900b-d530-44a1-8e77-d6b412fe2f65-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b\" (UID: \"7b43900b-d530-44a1-8e77-d6b412fe2f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"
Aug 05 18:05:58.645455 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.645433    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b43900b-d530-44a1-8e77-d6b412fe2f65-maas-api-tls\") pod \"maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b\" (UID: \"7b43900b-d530-44a1-8e77-d6b412fe2f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"
Aug 05 18:05:58.651419 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.651390    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w257j\" (UniqueName: \"kubernetes.io/projected/7b43900b-d530-44a1-8e77-d6b412fe2f65-kube-api-access-w257j\") pod \"maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b\" (UID: \"7b43900b-d530-44a1-8e77-d6b412fe2f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"
Aug 05 18:05:58.691888 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.691862    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"
Aug 05 18:05:58.827897 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:58.827868    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"]
Aug 05 18:05:58.830223 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:05:58.830197    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b43900b_d530_44a1_8e77_d6b412fe2f65.slice/crio-1fc51846c119dd26e5416cdfd5d9cf5a1c4b667251666923684bbe03c66403d6 WatchSource:0}: Error finding container 1fc51846c119dd26e5416cdfd5d9cf5a1c4b667251666923684bbe03c66403d6: Status 404 returned error can't find the container with id 1fc51846c119dd26e5416cdfd5d9cf5a1c4b667251666923684bbe03c66403d6
Aug 05 18:05:59.147601 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.147565    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"]
Aug 05 18:05:59.151433 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.151416    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"
Aug 05 18:05:59.153855 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.153836    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-3f8e0d-dockercfg-8hgrc\""
Aug 05 18:05:59.153970 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.153859    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-3f8e0d\""
Aug 05 18:05:59.161379 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.161352    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"]
Aug 05 18:05:59.236872 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.236843    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"]
Aug 05 18:05:59.239708 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.239689    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"
Aug 05 18:05:59.248060 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.248039    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"]
Aug 05 18:05:59.349431 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.349396    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqm7b\" (UniqueName: \"kubernetes.io/projected/384cf89a-b510-478d-9411-85a3f89d7fc7-kube-api-access-jqm7b\") pod \"payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v\" (UID: \"384cf89a-b510-478d-9411-85a3f89d7fc7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"
Aug 05 18:05:59.349675 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.349652    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f79e26a-9c6c-4347-8088-2356b2be9445-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c\" (UID: \"1f79e26a-9c6c-4347-8088-2356b2be9445\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"
Aug 05 18:05:59.349818 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.349802    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/384cf89a-b510-478d-9411-85a3f89d7fc7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v\" (UID: \"384cf89a-b510-478d-9411-85a3f89d7fc7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"
Aug 05 18:05:59.350038 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.350021    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lrql\" (UniqueName: \"kubernetes.io/projected/1f79e26a-9c6c-4347-8088-2356b2be9445-kube-api-access-6lrql\") pod \"payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c\" (UID: \"1f79e26a-9c6c-4347-8088-2356b2be9445\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"
Aug 05 18:05:59.451325 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.451297    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lrql\" (UniqueName: \"kubernetes.io/projected/1f79e26a-9c6c-4347-8088-2356b2be9445-kube-api-access-6lrql\") pod \"payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c\" (UID: \"1f79e26a-9c6c-4347-8088-2356b2be9445\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"
Aug 05 18:05:59.451690 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.451358    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqm7b\" (UniqueName: \"kubernetes.io/projected/384cf89a-b510-478d-9411-85a3f89d7fc7-kube-api-access-jqm7b\") pod \"payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v\" (UID: \"384cf89a-b510-478d-9411-85a3f89d7fc7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"
Aug 05 18:05:59.451690 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.451393    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f79e26a-9c6c-4347-8088-2356b2be9445-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c\" (UID: \"1f79e26a-9c6c-4347-8088-2356b2be9445\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"
Aug 05 18:05:59.451690 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.451421    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/384cf89a-b510-478d-9411-85a3f89d7fc7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v\" (UID: \"384cf89a-b510-478d-9411-85a3f89d7fc7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"
Aug 05 18:05:59.452112 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.452094    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/384cf89a-b510-478d-9411-85a3f89d7fc7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v\" (UID: \"384cf89a-b510-478d-9411-85a3f89d7fc7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"
Aug 05 18:05:59.452217 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.452199    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f79e26a-9c6c-4347-8088-2356b2be9445-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c\" (UID: \"1f79e26a-9c6c-4347-8088-2356b2be9445\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"
Aug 05 18:05:59.459907 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.459888    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqm7b\" (UniqueName: \"kubernetes.io/projected/384cf89a-b510-478d-9411-85a3f89d7fc7-kube-api-access-jqm7b\") pod \"payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v\" (UID: \"384cf89a-b510-478d-9411-85a3f89d7fc7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"
Aug 05 18:05:59.460114 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.460098    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lrql\" (UniqueName: \"kubernetes.io/projected/1f79e26a-9c6c-4347-8088-2356b2be9445-kube-api-access-6lrql\") pod \"payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c\" (UID: \"1f79e26a-9c6c-4347-8088-2356b2be9445\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"
Aug 05 18:05:59.462893 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.462874    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"
Aug 05 18:05:59.488087 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.488057    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b" event={"ID":"7b43900b-d530-44a1-8e77-d6b412fe2f65","Type":"ContainerStarted","Data":"cf3789a3e8c56bcbfdbd0f2291e0e218bd1612f7d79e67e018e816184fa9032a"}
Aug 05 18:05:59.488225 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.488167    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"
Aug 05 18:05:59.488225 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.488194    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b" event={"ID":"7b43900b-d530-44a1-8e77-d6b412fe2f65","Type":"ContainerStarted","Data":"1fc51846c119dd26e5416cdfd5d9cf5a1c4b667251666923684bbe03c66403d6"}
Aug 05 18:05:59.505263 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.505220    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b" podStartSLOduration=0.954598446 podStartE2EDuration="1.505205648s" totalImagesPullingTime="550.607202ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:05:59.503908146 +0000 UTC m=+2772.922222931" watchObservedRunningTime="2026-08-05 18:05:59.505205648 +0000 UTC m=+2772.923520404"
Aug 05 18:05:59.551581 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.551539    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"
Aug 05 18:05:59.599459 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.599431    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"]
Aug 05 18:05:59.602075 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:05:59.602043    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f79e26a_9c6c_4347_8088_2356b2be9445.slice/crio-613399f2a7e85a27980ee66846c8a82a79ad01011160c6423c07dc4606b1053e WatchSource:0}: Error finding container 613399f2a7e85a27980ee66846c8a82a79ad01011160c6423c07dc4606b1053e: Status 404 returned error can't find the container with id 613399f2a7e85a27980ee66846c8a82a79ad01011160c6423c07dc4606b1053e
Aug 05 18:05:59.702126 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:05:59.702102    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"]
Aug 05 18:05:59.703959 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:05:59.703937    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod384cf89a_b510_478d_9411_85a3f89d7fc7.slice/crio-3aa481ba34be4d280a807897d0430eb85726278413429ca4d0135d1f069269cd WatchSource:0}: Error finding container 3aa481ba34be4d280a807897d0430eb85726278413429ca4d0135d1f069269cd: Status 404 returned error can't find the container with id 3aa481ba34be4d280a807897d0430eb85726278413429ca4d0135d1f069269cd
Aug 05 18:06:00.495468 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:00.495434    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c" event={"ID":"1f79e26a-9c6c-4347-8088-2356b2be9445","Type":"ContainerStarted","Data":"3e352b1c6bdafcccdf59e1932740b650eac462e381d3363abffdc86a4f9ac036"}
Aug 05 18:06:00.495926 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:00.495488    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c" event={"ID":"1f79e26a-9c6c-4347-8088-2356b2be9445","Type":"ContainerStarted","Data":"613399f2a7e85a27980ee66846c8a82a79ad01011160c6423c07dc4606b1053e"}
Aug 05 18:06:00.495926 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:00.495531    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"
Aug 05 18:06:00.497126 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:00.497102    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v" event={"ID":"384cf89a-b510-478d-9411-85a3f89d7fc7","Type":"ContainerStarted","Data":"4e54b323d4098f8aabf8c3ff9e49e21226582d8fa13fd6310e897afeb43f7bdb"}
Aug 05 18:06:00.497287 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:00.497273    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"
Aug 05 18:06:00.497354 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:00.497308    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v" event={"ID":"384cf89a-b510-478d-9411-85a3f89d7fc7","Type":"ContainerStarted","Data":"3aa481ba34be4d280a807897d0430eb85726278413429ca4d0135d1f069269cd"}
Aug 05 18:06:00.513401 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:00.513360    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c" podStartSLOduration=1.51334697 podStartE2EDuration="1.51334697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:05:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:06:00.511801089 +0000 UTC m=+2773.930115887" watchObservedRunningTime="2026-08-05 18:06:00.51334697 +0000 UTC m=+2773.931661727"
Aug 05 18:06:00.527651 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:00.527610    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v" podStartSLOduration=1.527596107 podStartE2EDuration="1.527596107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:05:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:06:00.526880185 +0000 UTC m=+2773.945194943" watchObservedRunningTime="2026-08-05 18:06:00.527596107 +0000 UTC m=+2773.945910863"
Aug 05 18:06:05.502990 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:05.502962    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"
Aug 05 18:06:11.505141 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:11.505071    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"
Aug 05 18:06:11.505587 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:11.505201    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"
Aug 05 18:06:26.356925 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:26.356887    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"]
Aug 05 18:06:26.360807 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:26.360788    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:06:26.363845 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:26.363823    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-30613e-serving-cert\""
Aug 05 18:06:26.374507 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:26.374485    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"]
Aug 05 18:06:26.459607 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:26.459582    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ced8f1bb-169d-4634-aa79-0664db0f9a23-openshift-service-ca\") pod \"maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8\" (UID: \"ced8f1bb-169d-4634-aa79-0664db0f9a23\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:06:26.459742 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:26.459627    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ced8f1bb-169d-4634-aa79-0664db0f9a23-maas-api-tls\") pod \"maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8\" (UID: \"ced8f1bb-169d-4634-aa79-0664db0f9a23\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:06:26.459742 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:26.459696    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bjpz\" (UniqueName: \"kubernetes.io/projected/ced8f1bb-169d-4634-aa79-0664db0f9a23-kube-api-access-7bjpz\") pod \"maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8\" (UID: \"ced8f1bb-169d-4634-aa79-0664db0f9a23\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:06:26.560386 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:26.560357    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ced8f1bb-169d-4634-aa79-0664db0f9a23-openshift-service-ca\") pod \"maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8\" (UID: \"ced8f1bb-169d-4634-aa79-0664db0f9a23\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:06:26.560538 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:26.560412    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ced8f1bb-169d-4634-aa79-0664db0f9a23-maas-api-tls\") pod \"maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8\" (UID: \"ced8f1bb-169d-4634-aa79-0664db0f9a23\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:06:26.560538 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:26.560441    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bjpz\" (UniqueName: \"kubernetes.io/projected/ced8f1bb-169d-4634-aa79-0664db0f9a23-kube-api-access-7bjpz\") pod \"maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8\" (UID: \"ced8f1bb-169d-4634-aa79-0664db0f9a23\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:06:26.560664 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:06:26.560576    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-serving-cert: secret "maas-api-e2e-shared-b-30613e-serving-cert" not found
Aug 05 18:06:26.560715 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:06:26.560684    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ced8f1bb-169d-4634-aa79-0664db0f9a23-maas-api-tls podName:ced8f1bb-169d-4634-aa79-0664db0f9a23 nodeName:}" failed. No retries permitted until 2026-08-05 18:06:27.060661678 +0000 UTC m=+2800.478976427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ced8f1bb-169d-4634-aa79-0664db0f9a23-maas-api-tls") pod "maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8" (UID: "ced8f1bb-169d-4634-aa79-0664db0f9a23") : secret "maas-api-e2e-shared-b-30613e-serving-cert" not found
Aug 05 18:06:26.560982 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:26.560964    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ced8f1bb-169d-4634-aa79-0664db0f9a23-openshift-service-ca\") pod \"maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8\" (UID: \"ced8f1bb-169d-4634-aa79-0664db0f9a23\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:06:26.568951 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:26.568929    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bjpz\" (UniqueName: \"kubernetes.io/projected/ced8f1bb-169d-4634-aa79-0664db0f9a23-kube-api-access-7bjpz\") pod \"maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8\" (UID: \"ced8f1bb-169d-4634-aa79-0664db0f9a23\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:06:26.998619 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:26.998589    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"]
Aug 05 18:06:27.002272 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.002252    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"
Aug 05 18:06:27.004645 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.004625    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-30613e-dockercfg-4lnmw\""
Aug 05 18:06:27.004803 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.004787    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-30613e\""
Aug 05 18:06:27.015685 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.015663    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"]
Aug 05 18:06:27.036551 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.036527    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"]
Aug 05 18:06:27.039771 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.039754    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"
Aug 05 18:06:27.052053 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.052030    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"]
Aug 05 18:06:27.064938 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.064906    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glkgk\" (UniqueName: \"kubernetes.io/projected/565726c0-7400-4c14-a167-b2197ac507b5-kube-api-access-glkgk\") pod \"payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp\" (UID: \"565726c0-7400-4c14-a167-b2197ac507b5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"
Aug 05 18:06:27.065038 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.064973    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ced8f1bb-169d-4634-aa79-0664db0f9a23-maas-api-tls\") pod \"maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8\" (UID: \"ced8f1bb-169d-4634-aa79-0664db0f9a23\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:06:27.065038 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.065004    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/565726c0-7400-4c14-a167-b2197ac507b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp\" (UID: \"565726c0-7400-4c14-a167-b2197ac507b5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"
Aug 05 18:06:27.067335 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.067316    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ced8f1bb-169d-4634-aa79-0664db0f9a23-maas-api-tls\") pod \"maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8\" (UID: \"ced8f1bb-169d-4634-aa79-0664db0f9a23\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:06:27.166014 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.165977    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glkgk\" (UniqueName: \"kubernetes.io/projected/565726c0-7400-4c14-a167-b2197ac507b5-kube-api-access-glkgk\") pod \"payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp\" (UID: \"565726c0-7400-4c14-a167-b2197ac507b5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"
Aug 05 18:06:27.166230 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.166026    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2prdj\" (UniqueName: \"kubernetes.io/projected/58bbb020-3921-4ed1-b646-eba06286b0de-kube-api-access-2prdj\") pod \"payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8\" (UID: \"58bbb020-3921-4ed1-b646-eba06286b0de\") " pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"
Aug 05 18:06:27.166230 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.166075    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58bbb020-3921-4ed1-b646-eba06286b0de-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8\" (UID: \"58bbb020-3921-4ed1-b646-eba06286b0de\") " pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"
Aug 05 18:06:27.166230 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.166103    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/565726c0-7400-4c14-a167-b2197ac507b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp\" (UID: \"565726c0-7400-4c14-a167-b2197ac507b5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"
Aug 05 18:06:27.166800 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.166781    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/565726c0-7400-4c14-a167-b2197ac507b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp\" (UID: \"565726c0-7400-4c14-a167-b2197ac507b5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"
Aug 05 18:06:27.174160 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.174113    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glkgk\" (UniqueName: \"kubernetes.io/projected/565726c0-7400-4c14-a167-b2197ac507b5-kube-api-access-glkgk\") pod \"payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp\" (UID: \"565726c0-7400-4c14-a167-b2197ac507b5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"
Aug 05 18:06:27.267462 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.267359    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2prdj\" (UniqueName: \"kubernetes.io/projected/58bbb020-3921-4ed1-b646-eba06286b0de-kube-api-access-2prdj\") pod \"payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8\" (UID: \"58bbb020-3921-4ed1-b646-eba06286b0de\") " pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"
Aug 05 18:06:27.267462 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.267451    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58bbb020-3921-4ed1-b646-eba06286b0de-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8\" (UID: \"58bbb020-3921-4ed1-b646-eba06286b0de\") " pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"
Aug 05 18:06:27.268318 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.268294    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58bbb020-3921-4ed1-b646-eba06286b0de-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8\" (UID: \"58bbb020-3921-4ed1-b646-eba06286b0de\") " pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"
Aug 05 18:06:27.271194 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.271173    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:06:27.275185 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.275162    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2prdj\" (UniqueName: \"kubernetes.io/projected/58bbb020-3921-4ed1-b646-eba06286b0de-kube-api-access-2prdj\") pod \"payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8\" (UID: \"58bbb020-3921-4ed1-b646-eba06286b0de\") " pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"
Aug 05 18:06:27.314294 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.314260    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"
Aug 05 18:06:27.349288 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.349246    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"
Aug 05 18:06:27.427704 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.427363    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"]
Aug 05 18:06:27.429518 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:06:27.429419    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podced8f1bb_169d_4634_aa79_0664db0f9a23.slice/crio-77c115e70b029dbe28c50938d7d610f8391f45ff038f29ec9a290ae4b21b509c WatchSource:0}: Error finding container 77c115e70b029dbe28c50938d7d610f8391f45ff038f29ec9a290ae4b21b509c: Status 404 returned error can't find the container with id 77c115e70b029dbe28c50938d7d610f8391f45ff038f29ec9a290ae4b21b509c
Aug 05 18:06:27.485333 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.485237    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"]
Aug 05 18:06:27.488377 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:06:27.488350    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod565726c0_7400_4c14_a167_b2197ac507b5.slice/crio-dba259accccf6c351e2f40cdf5bdeb34c93e57bf6990807ab2203faccb0caaae WatchSource:0}: Error finding container dba259accccf6c351e2f40cdf5bdeb34c93e57bf6990807ab2203faccb0caaae: Status 404 returned error can't find the container with id dba259accccf6c351e2f40cdf5bdeb34c93e57bf6990807ab2203faccb0caaae
Aug 05 18:06:27.627328 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.627289    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8" event={"ID":"ced8f1bb-169d-4634-aa79-0664db0f9a23","Type":"ContainerStarted","Data":"77c115e70b029dbe28c50938d7d610f8391f45ff038f29ec9a290ae4b21b509c"}
Aug 05 18:06:27.628859 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.628837    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp" event={"ID":"565726c0-7400-4c14-a167-b2197ac507b5","Type":"ContainerStarted","Data":"6f24c3fee2e88b2d199bf80fc115d5e0ace9d7931759f7f62f67d9dc619923fb"}
Aug 05 18:06:27.628986 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.628875    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp" event={"ID":"565726c0-7400-4c14-a167-b2197ac507b5","Type":"ContainerStarted","Data":"dba259accccf6c351e2f40cdf5bdeb34c93e57bf6990807ab2203faccb0caaae"}
Aug 05 18:06:27.629039 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.629020    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"
Aug 05 18:06:27.644775 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.644729    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp" podStartSLOduration=1.64471358 podStartE2EDuration="1.64471358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:06:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:06:27.644000236 +0000 UTC m=+2801.062314994" watchObservedRunningTime="2026-08-05 18:06:27.64471358 +0000 UTC m=+2801.063028377"
Aug 05 18:06:27.722704 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:27.722681    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"]
Aug 05 18:06:27.724590 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:06:27.724556    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58bbb020_3921_4ed1_b646_eba06286b0de.slice/crio-6a58c1c3cea70fa17e8727f0d0c6611ddb51aecd4e9c2297686a5b326fd52b7b WatchSource:0}: Error finding container 6a58c1c3cea70fa17e8727f0d0c6611ddb51aecd4e9c2297686a5b326fd52b7b: Status 404 returned error can't find the container with id 6a58c1c3cea70fa17e8727f0d0c6611ddb51aecd4e9c2297686a5b326fd52b7b
Aug 05 18:06:28.635579 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:28.635536    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8" event={"ID":"58bbb020-3921-4ed1-b646-eba06286b0de","Type":"ContainerStarted","Data":"f9a32d995cc4adac56905997451ae85589a265e026cf624a371a02b49c2ec009"}
Aug 05 18:06:28.636037 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:28.635610    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"
Aug 05 18:06:28.636037 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:28.635632    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8" event={"ID":"58bbb020-3921-4ed1-b646-eba06286b0de","Type":"ContainerStarted","Data":"6a58c1c3cea70fa17e8727f0d0c6611ddb51aecd4e9c2297686a5b326fd52b7b"}
Aug 05 18:06:28.637049 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:28.637027    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8" event={"ID":"ced8f1bb-169d-4634-aa79-0664db0f9a23","Type":"ContainerStarted","Data":"34891834a91f59aab7d440e790aef47639ac550e3ea7b84f0e53b9390cfbc0f8"}
Aug 05 18:06:28.637202 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:28.637121    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:06:28.652198 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:28.652161    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8" podStartSLOduration=1.65213666 podStartE2EDuration="1.65213666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:06:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:06:28.650734059 +0000 UTC m=+2802.069048832" watchObservedRunningTime="2026-08-05 18:06:28.65213666 +0000 UTC m=+2802.070451417"
Aug 05 18:06:28.668299 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:28.668260    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8" podStartSLOduration=2.112996029 podStartE2EDuration="2.668247459s" totalImagesPullingTime="555.25143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:06:28.666778776 +0000 UTC m=+2802.085093535" watchObservedRunningTime="2026-08-05 18:06:28.668247459 +0000 UTC m=+2802.086562234"
Aug 05 18:06:34.649239 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:34.649205    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:06:38.639762 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:38.639730    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"
Aug 05 18:06:39.645324 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:39.645291    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"
Aug 05 18:06:55.363097 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.363061    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"]
Aug 05 18:06:55.366789 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.366768    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.369315 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.369291    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3f8e0d\"/\"openshift-service-ca.crt\""
Aug 05 18:06:55.370409 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.370388    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3f8e0d\"/\"default-dockercfg-xpvxf\""
Aug 05 18:06:55.370519 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.370430    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3f8e0d\"/\"auth-test-model-3f8e0d-kserve-self-signed-certs\""
Aug 05 18:06:55.370519 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.370439    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3f8e0d\"/\"kube-root-ca.crt\""
Aug 05 18:06:55.374730 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.374594    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"]
Aug 05 18:06:55.509441 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.509410    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxjww\" (UniqueName: \"kubernetes.io/projected/3205ca72-392d-4bc6-b487-98ecead8f588-kube-api-access-zxjww\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.509647 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.509478    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-dshm\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.509647 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.509513    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-model-cache\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.509647 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.509528    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-tmp-dir\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.509647 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.509555    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3205ca72-392d-4bc6-b487-98ecead8f588-tls-certs\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.509647 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.509635    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-home\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.610599 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.610558    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-model-cache\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.610599 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.610604    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-tmp-dir\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.610849 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.610668    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3205ca72-392d-4bc6-b487-98ecead8f588-tls-certs\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.610849 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.610719    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-home\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.610849 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.610804    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxjww\" (UniqueName: \"kubernetes.io/projected/3205ca72-392d-4bc6-b487-98ecead8f588-kube-api-access-zxjww\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.611013 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.610896    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-dshm\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.611068 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.611015    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-tmp-dir\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.611123 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.611109    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-home\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.611387 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.611367    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-model-cache\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.613042 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.613014    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-dshm\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.613384 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.613333    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3205ca72-392d-4bc6-b487-98ecead8f588-tls-certs\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.618762 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.618734    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxjww\" (UniqueName: \"kubernetes.io/projected/3205ca72-392d-4bc6-b487-98ecead8f588-kube-api-access-zxjww\") pod \"auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") " pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.677752 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.677711    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:06:55.806484 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:55.806457    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"]
Aug 05 18:06:55.809040 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:06:55.809008    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3205ca72_392d_4bc6_b487_98ecead8f588.slice/crio-1f43acc8550d55e2784f0dfef022d448bdd91f96b6728034be4e12bc774522e5 WatchSource:0}: Error finding container 1f43acc8550d55e2784f0dfef022d448bdd91f96b6728034be4e12bc774522e5: Status 404 returned error can't find the container with id 1f43acc8550d55e2784f0dfef022d448bdd91f96b6728034be4e12bc774522e5
Aug 05 18:06:56.773351 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:56.773320    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth" event={"ID":"3205ca72-392d-4bc6-b487-98ecead8f588","Type":"ContainerStarted","Data":"9528f6f802fced0c4ef49965e94a779549aff2c21900c3dc005109354727d410"}
Aug 05 18:06:56.773719 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:56.773361    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth" event={"ID":"3205ca72-392d-4bc6-b487-98ecead8f588","Type":"ContainerStarted","Data":"1f43acc8550d55e2784f0dfef022d448bdd91f96b6728034be4e12bc774522e5"}
Aug 05 18:06:56.794665 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:06:56.794622    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth" podStartSLOduration=1.7946071749999999 podStartE2EDuration="1.794607175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:06:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:06:56.791998745 +0000 UTC m=+2830.210313493" watchObservedRunningTime="2026-08-05 18:06:56.794607175 +0000 UTC m=+2830.212921931"
Aug 05 18:07:05.677864 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:07:05.677830    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:07:05.678381 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:07:05.677877    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:07:05.690198 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:07:05.690175    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:07:05.825704 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:07:05.825675    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:10:15.699042 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:15.699009    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"]
Aug 05 18:10:30.718552 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:30.718510    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth" podUID="3205ca72-392d-4bc6-b487-98ecead8f588" containerName="main" containerID="cri-o://9528f6f802fced0c4ef49965e94a779549aff2c21900c3dc005109354727d410" gracePeriod=45
Aug 05 18:10:30.968338 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:30.968306    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:10:31.094065 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.093989    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-dshm\") pod \"3205ca72-392d-4bc6-b487-98ecead8f588\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") "
Aug 05 18:10:31.094237 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.094065    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3205ca72-392d-4bc6-b487-98ecead8f588-tls-certs\") pod \"3205ca72-392d-4bc6-b487-98ecead8f588\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") "
Aug 05 18:10:31.094237 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.094094    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-home\") pod \"3205ca72-392d-4bc6-b487-98ecead8f588\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") "
Aug 05 18:10:31.094237 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.094128    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxjww\" (UniqueName: \"kubernetes.io/projected/3205ca72-392d-4bc6-b487-98ecead8f588-kube-api-access-zxjww\") pod \"3205ca72-392d-4bc6-b487-98ecead8f588\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") "
Aug 05 18:10:31.094237 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.094191    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-model-cache\") pod \"3205ca72-392d-4bc6-b487-98ecead8f588\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") "
Aug 05 18:10:31.094472 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.094401    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-home" (OuterVolumeSpecName: "home") pod "3205ca72-392d-4bc6-b487-98ecead8f588" (UID: "3205ca72-392d-4bc6-b487-98ecead8f588"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:10:31.094472 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.094451    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-model-cache" (OuterVolumeSpecName: "model-cache") pod "3205ca72-392d-4bc6-b487-98ecead8f588" (UID: "3205ca72-392d-4bc6-b487-98ecead8f588"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:10:31.094579 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.094497    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-tmp-dir\") pod \"3205ca72-392d-4bc6-b487-98ecead8f588\" (UID: \"3205ca72-392d-4bc6-b487-98ecead8f588\") "
Aug 05 18:10:31.094687 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.094662    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3205ca72-392d-4bc6-b487-98ecead8f588" (UID: "3205ca72-392d-4bc6-b487-98ecead8f588"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:10:31.094874 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.094855    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-home\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:10:31.094953 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.094878    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-model-cache\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:10:31.094953 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.094916    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-tmp-dir\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:10:31.096388 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.096357    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3205ca72-392d-4bc6-b487-98ecead8f588-kube-api-access-zxjww" (OuterVolumeSpecName: "kube-api-access-zxjww") pod "3205ca72-392d-4bc6-b487-98ecead8f588" (UID: "3205ca72-392d-4bc6-b487-98ecead8f588"). InnerVolumeSpecName "kube-api-access-zxjww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:10:31.096504 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.096436    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-dshm" (OuterVolumeSpecName: "dshm") pod "3205ca72-392d-4bc6-b487-98ecead8f588" (UID: "3205ca72-392d-4bc6-b487-98ecead8f588"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:10:31.096567 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.096517    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3205ca72-392d-4bc6-b487-98ecead8f588-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3205ca72-392d-4bc6-b487-98ecead8f588" (UID: "3205ca72-392d-4bc6-b487-98ecead8f588"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:10:31.195453 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.195426    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3205ca72-392d-4bc6-b487-98ecead8f588-dshm\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:10:31.195453 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.195447    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3205ca72-392d-4bc6-b487-98ecead8f588-tls-certs\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:10:31.195597 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.195458    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxjww\" (UniqueName: \"kubernetes.io/projected/3205ca72-392d-4bc6-b487-98ecead8f588-kube-api-access-zxjww\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:10:31.834309 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.834271    2556 generic.go:356] "Generic (PLEG): container finished" podID="3205ca72-392d-4bc6-b487-98ecead8f588" containerID="9528f6f802fced0c4ef49965e94a779549aff2c21900c3dc005109354727d410" exitCode=0
Aug 05 18:10:31.834309 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.834312    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"
Aug 05 18:10:31.834857 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.834357    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth" event={"ID":"3205ca72-392d-4bc6-b487-98ecead8f588","Type":"ContainerDied","Data":"9528f6f802fced0c4ef49965e94a779549aff2c21900c3dc005109354727d410"}
Aug 05 18:10:31.834857 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.834403    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth" event={"ID":"3205ca72-392d-4bc6-b487-98ecead8f588","Type":"ContainerDied","Data":"1f43acc8550d55e2784f0dfef022d448bdd91f96b6728034be4e12bc774522e5"}
Aug 05 18:10:31.834857 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.834421    2556 scope.go:117] "RemoveContainer" containerID="9528f6f802fced0c4ef49965e94a779549aff2c21900c3dc005109354727d410"
Aug 05 18:10:31.842924 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.842908    2556 scope.go:117] "RemoveContainer" containerID="9528f6f802fced0c4ef49965e94a779549aff2c21900c3dc005109354727d410"
Aug 05 18:10:31.843190 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:10:31.843171    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9528f6f802fced0c4ef49965e94a779549aff2c21900c3dc005109354727d410\": container with ID starting with 9528f6f802fced0c4ef49965e94a779549aff2c21900c3dc005109354727d410 not found: ID does not exist" containerID="9528f6f802fced0c4ef49965e94a779549aff2c21900c3dc005109354727d410"
Aug 05 18:10:31.843243 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.843198    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9528f6f802fced0c4ef49965e94a779549aff2c21900c3dc005109354727d410"} err="failed to get container status \"9528f6f802fced0c4ef49965e94a779549aff2c21900c3dc005109354727d410\": rpc error: code = NotFound desc = could not find container \"9528f6f802fced0c4ef49965e94a779549aff2c21900c3dc005109354727d410\": container with ID starting with 9528f6f802fced0c4ef49965e94a779549aff2c21900c3dc005109354727d410 not found: ID does not exist"
Aug 05 18:10:31.850001 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.849978    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"]
Aug 05 18:10:31.856173 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:31.856126    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3f8e0d/auth-test-model-3f8e0d-kserve-6494ddb4f9-dkjth"]
Aug 05 18:10:33.097289 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:33.097258    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3205ca72-392d-4bc6-b487-98ecead8f588" path="/var/lib/kubelet/pods/3205ca72-392d-4bc6-b487-98ecead8f588/volumes"
Aug 05 18:10:45.971399 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:45.971306    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"]
Aug 05 18:10:45.972020 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:45.971982    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3205ca72-392d-4bc6-b487-98ecead8f588" containerName="main"
Aug 05 18:10:45.972108 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:45.972003    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3205ca72-392d-4bc6-b487-98ecead8f588" containerName="main"
Aug 05 18:10:45.972197 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:45.972165    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3205ca72-392d-4bc6-b487-98ecead8f588" containerName="main"
Aug 05 18:10:45.976782 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:45.976763    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:45.980400 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:45.980375    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-30613e\"/\"default-dockercfg-5zqkf\""
Aug 05 18:10:45.980499 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:45.980401    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-30613e\"/\"sub-test-model-30613e-kserve-self-signed-certs\""
Aug 05 18:10:45.980694 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:45.980638    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-30613e\"/\"kube-root-ca.crt\""
Aug 05 18:10:45.980694 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:45.980658    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-30613e\"/\"openshift-service-ca.crt\""
Aug 05 18:10:45.985357 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:45.985335    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"]
Aug 05 18:10:46.113507 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.113476    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-tmp-dir\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.113697 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.113541    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-home\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.113697 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.113583    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-model-cache\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.113697 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.113606    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-dshm\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.113697 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.113684    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr6hx\" (UniqueName: \"kubernetes.io/projected/c47fd51f-d317-4899-b831-3a0b66304fb3-kube-api-access-vr6hx\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.113980 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.113732    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c47fd51f-d317-4899-b831-3a0b66304fb3-tls-certs\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.214928 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.214888    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-home\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.214928 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.214929    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-model-cache\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.215200 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.214953    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-dshm\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.215200 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.214987    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr6hx\" (UniqueName: \"kubernetes.io/projected/c47fd51f-d317-4899-b831-3a0b66304fb3-kube-api-access-vr6hx\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.215200 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.215019    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c47fd51f-d317-4899-b831-3a0b66304fb3-tls-certs\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.215200 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.215070    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-tmp-dir\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.215424 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.215379    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-model-cache\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.215480 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.215433    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-home\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.215549 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.215499    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-tmp-dir\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.217336 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.217311    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-dshm\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.217569 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.217551    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c47fd51f-d317-4899-b831-3a0b66304fb3-tls-certs\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.222313 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.222262    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr6hx\" (UniqueName: \"kubernetes.io/projected/c47fd51f-d317-4899-b831-3a0b66304fb3-kube-api-access-vr6hx\") pod \"sub-test-model-30613e-kserve-77474854f9-gm5gn\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") " pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.286872 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.286844    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:46.417795 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.417768    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"]
Aug 05 18:10:46.420067 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:10:46.420039    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc47fd51f_d317_4899_b831_3a0b66304fb3.slice/crio-90d996cad21ceb6cb5be1a9ddf708cebd92b2794a030187472e725e1a1e453eb WatchSource:0}: Error finding container 90d996cad21ceb6cb5be1a9ddf708cebd92b2794a030187472e725e1a1e453eb: Status 404 returned error can't find the container with id 90d996cad21ceb6cb5be1a9ddf708cebd92b2794a030187472e725e1a1e453eb
Aug 05 18:10:46.910565 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.910470    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn" event={"ID":"c47fd51f-d317-4899-b831-3a0b66304fb3","Type":"ContainerStarted","Data":"39851b46b74eb9da03a7f44caccd6643b93d8aab9ce0c98d16a286541dc3fb59"}
Aug 05 18:10:46.910565 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.910528    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn" event={"ID":"c47fd51f-d317-4899-b831-3a0b66304fb3","Type":"ContainerStarted","Data":"90d996cad21ceb6cb5be1a9ddf708cebd92b2794a030187472e725e1a1e453eb"}
Aug 05 18:10:46.928592 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:46.928541    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn" podStartSLOduration=1.928524861 podStartE2EDuration="1.928524861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:10:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:10:46.92744162 +0000 UTC m=+3060.345756390" watchObservedRunningTime="2026-08-05 18:10:46.928524861 +0000 UTC m=+3060.346839618"
Aug 05 18:10:56.287233 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:56.287197    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:56.287593 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:56.287246    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:56.299652 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:56.299625    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:10:56.970184 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:10:56.970135    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:11:52.974091 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:11:52.974052    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"]
Aug 05 18:12:07.993006 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:07.992962    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn" podUID="c47fd51f-d317-4899-b831-3a0b66304fb3" containerName="main" containerID="cri-o://39851b46b74eb9da03a7f44caccd6643b93d8aab9ce0c98d16a286541dc3fb59" gracePeriod=45
Aug 05 18:12:08.256000 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.255944    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:12:08.294572 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.294535    2556 generic.go:356] "Generic (PLEG): container finished" podID="c47fd51f-d317-4899-b831-3a0b66304fb3" containerID="39851b46b74eb9da03a7f44caccd6643b93d8aab9ce0c98d16a286541dc3fb59" exitCode=0
Aug 05 18:12:08.294714 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.294575    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"
Aug 05 18:12:08.294714 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.294608    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn" event={"ID":"c47fd51f-d317-4899-b831-3a0b66304fb3","Type":"ContainerDied","Data":"39851b46b74eb9da03a7f44caccd6643b93d8aab9ce0c98d16a286541dc3fb59"}
Aug 05 18:12:08.294714 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.294645    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn" event={"ID":"c47fd51f-d317-4899-b831-3a0b66304fb3","Type":"ContainerDied","Data":"90d996cad21ceb6cb5be1a9ddf708cebd92b2794a030187472e725e1a1e453eb"}
Aug 05 18:12:08.294714 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.294665    2556 scope.go:117] "RemoveContainer" containerID="39851b46b74eb9da03a7f44caccd6643b93d8aab9ce0c98d16a286541dc3fb59"
Aug 05 18:12:08.303017 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.303001    2556 scope.go:117] "RemoveContainer" containerID="39851b46b74eb9da03a7f44caccd6643b93d8aab9ce0c98d16a286541dc3fb59"
Aug 05 18:12:08.303306 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:12:08.303288    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39851b46b74eb9da03a7f44caccd6643b93d8aab9ce0c98d16a286541dc3fb59\": container with ID starting with 39851b46b74eb9da03a7f44caccd6643b93d8aab9ce0c98d16a286541dc3fb59 not found: ID does not exist" containerID="39851b46b74eb9da03a7f44caccd6643b93d8aab9ce0c98d16a286541dc3fb59"
Aug 05 18:12:08.303370 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.303314    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39851b46b74eb9da03a7f44caccd6643b93d8aab9ce0c98d16a286541dc3fb59"} err="failed to get container status \"39851b46b74eb9da03a7f44caccd6643b93d8aab9ce0c98d16a286541dc3fb59\": rpc error: code = NotFound desc = could not find container \"39851b46b74eb9da03a7f44caccd6643b93d8aab9ce0c98d16a286541dc3fb59\": container with ID starting with 39851b46b74eb9da03a7f44caccd6643b93d8aab9ce0c98d16a286541dc3fb59 not found: ID does not exist"
Aug 05 18:12:08.320631 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.320610    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-dshm\") pod \"c47fd51f-d317-4899-b831-3a0b66304fb3\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") "
Aug 05 18:12:08.320740 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.320667    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-model-cache\") pod \"c47fd51f-d317-4899-b831-3a0b66304fb3\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") "
Aug 05 18:12:08.320740 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.320701    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vr6hx\" (UniqueName: \"kubernetes.io/projected/c47fd51f-d317-4899-b831-3a0b66304fb3-kube-api-access-vr6hx\") pod \"c47fd51f-d317-4899-b831-3a0b66304fb3\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") "
Aug 05 18:12:08.320859 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.320737    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-tmp-dir\") pod \"c47fd51f-d317-4899-b831-3a0b66304fb3\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") "
Aug 05 18:12:08.320859 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.320785    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-home\") pod \"c47fd51f-d317-4899-b831-3a0b66304fb3\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") "
Aug 05 18:12:08.320859 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.320850    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c47fd51f-d317-4899-b831-3a0b66304fb3-tls-certs\") pod \"c47fd51f-d317-4899-b831-3a0b66304fb3\" (UID: \"c47fd51f-d317-4899-b831-3a0b66304fb3\") "
Aug 05 18:12:08.321080 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.321049    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c47fd51f-d317-4899-b831-3a0b66304fb3" (UID: "c47fd51f-d317-4899-b831-3a0b66304fb3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:12:08.321146 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.321069    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-home" (OuterVolumeSpecName: "home") pod "c47fd51f-d317-4899-b831-3a0b66304fb3" (UID: "c47fd51f-d317-4899-b831-3a0b66304fb3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:12:08.321233 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.321139    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-model-cache" (OuterVolumeSpecName: "model-cache") pod "c47fd51f-d317-4899-b831-3a0b66304fb3" (UID: "c47fd51f-d317-4899-b831-3a0b66304fb3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:12:08.322968 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.322940    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c47fd51f-d317-4899-b831-3a0b66304fb3-kube-api-access-vr6hx" (OuterVolumeSpecName: "kube-api-access-vr6hx") pod "c47fd51f-d317-4899-b831-3a0b66304fb3" (UID: "c47fd51f-d317-4899-b831-3a0b66304fb3"). InnerVolumeSpecName "kube-api-access-vr6hx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:12:08.323056 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.322951    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-dshm" (OuterVolumeSpecName: "dshm") pod "c47fd51f-d317-4899-b831-3a0b66304fb3" (UID: "c47fd51f-d317-4899-b831-3a0b66304fb3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:12:08.323056 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.322967    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c47fd51f-d317-4899-b831-3a0b66304fb3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c47fd51f-d317-4899-b831-3a0b66304fb3" (UID: "c47fd51f-d317-4899-b831-3a0b66304fb3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:12:08.421889 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.421864    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-model-cache\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:12:08.421889 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.421887    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vr6hx\" (UniqueName: \"kubernetes.io/projected/c47fd51f-d317-4899-b831-3a0b66304fb3-kube-api-access-vr6hx\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:12:08.422040 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.421898    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-tmp-dir\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:12:08.422040 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.421908    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-home\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:12:08.422040 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.421916    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c47fd51f-d317-4899-b831-3a0b66304fb3-tls-certs\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:12:08.422040 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.421924    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c47fd51f-d317-4899-b831-3a0b66304fb3-dshm\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:12:08.615728 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.615697    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"]
Aug 05 18:12:08.617497 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:08.617474    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-30613e/sub-test-model-30613e-kserve-77474854f9-gm5gn"]
Aug 05 18:12:09.103411 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:09.103367    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c47fd51f-d317-4899-b831-3a0b66304fb3" path="/var/lib/kubelet/pods/c47fd51f-d317-4899-b831-3a0b66304fb3/volumes"
Aug 05 18:12:27.061004 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.060971    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"]
Aug 05 18:12:27.061606 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.061572    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c47fd51f-d317-4899-b831-3a0b66304fb3" containerName="main"
Aug 05 18:12:27.061606 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.061609    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c47fd51f-d317-4899-b831-3a0b66304fb3" containerName="main"
Aug 05 18:12:27.061823 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.061806    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c47fd51f-d317-4899-b831-3a0b66304fb3" containerName="main"
Aug 05 18:12:27.066381 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.066364    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.069991 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.069968    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-30613e\"/\"rate-test-model-30613e-kserve-self-signed-certs\""
Aug 05 18:12:27.069991 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.069990    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-30613e\"/\"default-dockercfg-5zqkf\""
Aug 05 18:12:27.070192 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.070028    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-30613e\"/\"openshift-service-ca.crt\""
Aug 05 18:12:27.070192 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.069979    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-30613e\"/\"kube-root-ca.crt\""
Aug 05 18:12:27.072723 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.072704    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"]
Aug 05 18:12:27.173232 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.173200    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-home\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.173394 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.173251    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-dshm\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.173394 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.173293    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab8b58c3-5530-4f7f-894b-78663dc2305c-tls-certs\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.173394 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.173334    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64n7h\" (UniqueName: \"kubernetes.io/projected/ab8b58c3-5530-4f7f-894b-78663dc2305c-kube-api-access-64n7h\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.173394 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.173363    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-model-cache\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.173528 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.173393    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-tmp-dir\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.274691 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.274652    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-home\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.274845 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.274714    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-dshm\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.274845 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.274737    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab8b58c3-5530-4f7f-894b-78663dc2305c-tls-certs\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.274845 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.274760    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64n7h\" (UniqueName: \"kubernetes.io/projected/ab8b58c3-5530-4f7f-894b-78663dc2305c-kube-api-access-64n7h\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.274845 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.274779    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-model-cache\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.274845 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.274807    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-tmp-dir\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.275126 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.275103    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-home\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.275248 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.275225    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-tmp-dir\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.275248 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.275240    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-model-cache\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.277063 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.277035    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-dshm\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.277358 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.277339    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab8b58c3-5530-4f7f-894b-78663dc2305c-tls-certs\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.285082 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.285049    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64n7h\" (UniqueName: \"kubernetes.io/projected/ab8b58c3-5530-4f7f-894b-78663dc2305c-kube-api-access-64n7h\") pod \"rate-test-model-30613e-kserve-7bb566c475-svh9j\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") " pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.376259 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.376204    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:27.512723 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:27.512694    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"]
Aug 05 18:12:27.514661 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:12:27.514631    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab8b58c3_5530_4f7f_894b_78663dc2305c.slice/crio-4f1c3124fa3e2113c3c68020dc27017aa9e874b3ceb3b271eea1452493e1e6fe WatchSource:0}: Error finding container 4f1c3124fa3e2113c3c68020dc27017aa9e874b3ceb3b271eea1452493e1e6fe: Status 404 returned error can't find the container with id 4f1c3124fa3e2113c3c68020dc27017aa9e874b3ceb3b271eea1452493e1e6fe
Aug 05 18:12:28.393618 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:28.393589    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j" event={"ID":"ab8b58c3-5530-4f7f-894b-78663dc2305c","Type":"ContainerStarted","Data":"ab6ea24d098fdcdf8fa1ca273150e2ef99c9dc32ed0d09448d9a5182ff0c4f97"}
Aug 05 18:12:28.393986 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:28.393631    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j" event={"ID":"ab8b58c3-5530-4f7f-894b-78663dc2305c","Type":"ContainerStarted","Data":"4f1c3124fa3e2113c3c68020dc27017aa9e874b3ceb3b271eea1452493e1e6fe"}
Aug 05 18:12:28.413662 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:28.413605    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j" podStartSLOduration=1.413584219 podStartE2EDuration="1.413584219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:12:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:12:28.410177799 +0000 UTC m=+3161.828492551" watchObservedRunningTime="2026-08-05 18:12:28.413584219 +0000 UTC m=+3161.831898977"
Aug 05 18:12:37.377218 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:37.377182    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:37.377673 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:37.377232    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:37.389895 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:37.389873    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:12:37.447180 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:12:37.447140    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:13:35.983120 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:35.983076    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"]
Aug 05 18:13:51.001751 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.001653    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j" podUID="ab8b58c3-5530-4f7f-894b-78663dc2305c" containerName="main" containerID="cri-o://ab6ea24d098fdcdf8fa1ca273150e2ef99c9dc32ed0d09448d9a5182ff0c4f97" gracePeriod=45
Aug 05 18:13:51.246544 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.246518    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:13:51.273701 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.273632    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-model-cache\") pod \"ab8b58c3-5530-4f7f-894b-78663dc2305c\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") "
Aug 05 18:13:51.273701 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.273694    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-home\") pod \"ab8b58c3-5530-4f7f-894b-78663dc2305c\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") "
Aug 05 18:13:51.273886 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.273727    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-dshm\") pod \"ab8b58c3-5530-4f7f-894b-78663dc2305c\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") "
Aug 05 18:13:51.273886 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.273752    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-tmp-dir\") pod \"ab8b58c3-5530-4f7f-894b-78663dc2305c\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") "
Aug 05 18:13:51.273886 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.273848    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab8b58c3-5530-4f7f-894b-78663dc2305c-tls-certs\") pod \"ab8b58c3-5530-4f7f-894b-78663dc2305c\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") "
Aug 05 18:13:51.274048 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.273884    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64n7h\" (UniqueName: \"kubernetes.io/projected/ab8b58c3-5530-4f7f-894b-78663dc2305c-kube-api-access-64n7h\") pod \"ab8b58c3-5530-4f7f-894b-78663dc2305c\" (UID: \"ab8b58c3-5530-4f7f-894b-78663dc2305c\") "
Aug 05 18:13:51.274048 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.273846    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-model-cache" (OuterVolumeSpecName: "model-cache") pod "ab8b58c3-5530-4f7f-894b-78663dc2305c" (UID: "ab8b58c3-5530-4f7f-894b-78663dc2305c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:13:51.274048 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.273946    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ab8b58c3-5530-4f7f-894b-78663dc2305c" (UID: "ab8b58c3-5530-4f7f-894b-78663dc2305c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:13:51.274048 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.273965    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-home" (OuterVolumeSpecName: "home") pod "ab8b58c3-5530-4f7f-894b-78663dc2305c" (UID: "ab8b58c3-5530-4f7f-894b-78663dc2305c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:13:51.274280 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.274172    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-model-cache\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:13:51.274280 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.274192    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-home\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:13:51.274280 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.274205    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-tmp-dir\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:13:51.275846 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.275816    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-dshm" (OuterVolumeSpecName: "dshm") pod "ab8b58c3-5530-4f7f-894b-78663dc2305c" (UID: "ab8b58c3-5530-4f7f-894b-78663dc2305c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:13:51.275957 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.275881    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab8b58c3-5530-4f7f-894b-78663dc2305c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ab8b58c3-5530-4f7f-894b-78663dc2305c" (UID: "ab8b58c3-5530-4f7f-894b-78663dc2305c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:13:51.276377 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.276353    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab8b58c3-5530-4f7f-894b-78663dc2305c-kube-api-access-64n7h" (OuterVolumeSpecName: "kube-api-access-64n7h") pod "ab8b58c3-5530-4f7f-894b-78663dc2305c" (UID: "ab8b58c3-5530-4f7f-894b-78663dc2305c"). InnerVolumeSpecName "kube-api-access-64n7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:13:51.375575 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.375551    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64n7h\" (UniqueName: \"kubernetes.io/projected/ab8b58c3-5530-4f7f-894b-78663dc2305c-kube-api-access-64n7h\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:13:51.375575 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.375572    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab8b58c3-5530-4f7f-894b-78663dc2305c-dshm\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:13:51.375731 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.375585    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab8b58c3-5530-4f7f-894b-78663dc2305c-tls-certs\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:13:51.800993 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.800958    2556 generic.go:356] "Generic (PLEG): container finished" podID="ab8b58c3-5530-4f7f-894b-78663dc2305c" containerID="ab6ea24d098fdcdf8fa1ca273150e2ef99c9dc32ed0d09448d9a5182ff0c4f97" exitCode=0
Aug 05 18:13:51.800993 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.800996    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"
Aug 05 18:13:51.801248 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.801040    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j" event={"ID":"ab8b58c3-5530-4f7f-894b-78663dc2305c","Type":"ContainerDied","Data":"ab6ea24d098fdcdf8fa1ca273150e2ef99c9dc32ed0d09448d9a5182ff0c4f97"}
Aug 05 18:13:51.801248 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.801084    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j" event={"ID":"ab8b58c3-5530-4f7f-894b-78663dc2305c","Type":"ContainerDied","Data":"4f1c3124fa3e2113c3c68020dc27017aa9e874b3ceb3b271eea1452493e1e6fe"}
Aug 05 18:13:51.801248 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.801107    2556 scope.go:117] "RemoveContainer" containerID="ab6ea24d098fdcdf8fa1ca273150e2ef99c9dc32ed0d09448d9a5182ff0c4f97"
Aug 05 18:13:51.809867 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.809847    2556 scope.go:117] "RemoveContainer" containerID="ab6ea24d098fdcdf8fa1ca273150e2ef99c9dc32ed0d09448d9a5182ff0c4f97"
Aug 05 18:13:51.810101 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:13:51.810082    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab6ea24d098fdcdf8fa1ca273150e2ef99c9dc32ed0d09448d9a5182ff0c4f97\": container with ID starting with ab6ea24d098fdcdf8fa1ca273150e2ef99c9dc32ed0d09448d9a5182ff0c4f97 not found: ID does not exist" containerID="ab6ea24d098fdcdf8fa1ca273150e2ef99c9dc32ed0d09448d9a5182ff0c4f97"
Aug 05 18:13:51.810174 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.810109    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab6ea24d098fdcdf8fa1ca273150e2ef99c9dc32ed0d09448d9a5182ff0c4f97"} err="failed to get container status \"ab6ea24d098fdcdf8fa1ca273150e2ef99c9dc32ed0d09448d9a5182ff0c4f97\": rpc error: code = NotFound desc = could not find container \"ab6ea24d098fdcdf8fa1ca273150e2ef99c9dc32ed0d09448d9a5182ff0c4f97\": container with ID starting with ab6ea24d098fdcdf8fa1ca273150e2ef99c9dc32ed0d09448d9a5182ff0c4f97 not found: ID does not exist"
Aug 05 18:13:51.822271 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.822248    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"]
Aug 05 18:13:51.826008 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:51.825988    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-30613e/rate-test-model-30613e-kserve-7bb566c475-svh9j"]
Aug 05 18:13:53.097090 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:13:53.097058    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab8b58c3-5530-4f7f-894b-78663dc2305c" path="/var/lib/kubelet/pods/ab8b58c3-5530-4f7f-894b-78663dc2305c/volumes"
Aug 05 18:14:01.278546 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.278514    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"]
Aug 05 18:14:01.278953 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.278918    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab8b58c3-5530-4f7f-894b-78663dc2305c" containerName="main"
Aug 05 18:14:01.278953 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.278929    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab8b58c3-5530-4f7f-894b-78663dc2305c" containerName="main"
Aug 05 18:14:01.279032 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.278999    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab8b58c3-5530-4f7f-894b-78663dc2305c" containerName="main"
Aug 05 18:14:01.281991 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.281972    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"
Aug 05 18:14:01.284550 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.284530    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-4babb1\""
Aug 05 18:14:01.291295 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.291274    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"]
Aug 05 18:14:01.305953 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.305928    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"]
Aug 05 18:14:01.321033 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.321009    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"]
Aug 05 18:14:01.321162 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.321103    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"
Aug 05 18:14:01.323504 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.323486    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-4babb1-dockercfg-g27jw\""
Aug 05 18:14:01.368474 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.368451    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d0bd32d-3d66-4dbd-be05-66d52a429d7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq\" (UID: \"4d0bd32d-3d66-4dbd-be05-66d52a429d7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"
Aug 05 18:14:01.368608 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.368499    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95zwr\" (UniqueName: \"kubernetes.io/projected/f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e-kube-api-access-95zwr\") pod \"payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv\" (UID: \"f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"
Aug 05 18:14:01.368685 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.368598    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv\" (UID: \"f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"
Aug 05 18:14:01.368685 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.368657    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dg2d\" (UniqueName: \"kubernetes.io/projected/4d0bd32d-3d66-4dbd-be05-66d52a429d7d-kube-api-access-5dg2d\") pod \"payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq\" (UID: \"4d0bd32d-3d66-4dbd-be05-66d52a429d7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"
Aug 05 18:14:01.469908 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.469873    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d0bd32d-3d66-4dbd-be05-66d52a429d7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq\" (UID: \"4d0bd32d-3d66-4dbd-be05-66d52a429d7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"
Aug 05 18:14:01.470095 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.469927    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95zwr\" (UniqueName: \"kubernetes.io/projected/f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e-kube-api-access-95zwr\") pod \"payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv\" (UID: \"f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"
Aug 05 18:14:01.470095 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.469981    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv\" (UID: \"f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"
Aug 05 18:14:01.470095 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.470009    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dg2d\" (UniqueName: \"kubernetes.io/projected/4d0bd32d-3d66-4dbd-be05-66d52a429d7d-kube-api-access-5dg2d\") pod \"payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq\" (UID: \"4d0bd32d-3d66-4dbd-be05-66d52a429d7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"
Aug 05 18:14:01.470719 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.470691    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d0bd32d-3d66-4dbd-be05-66d52a429d7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq\" (UID: \"4d0bd32d-3d66-4dbd-be05-66d52a429d7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"
Aug 05 18:14:01.470820 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.470791    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv\" (UID: \"f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"
Aug 05 18:14:01.477834 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.477812    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95zwr\" (UniqueName: \"kubernetes.io/projected/f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e-kube-api-access-95zwr\") pod \"payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv\" (UID: \"f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"
Aug 05 18:14:01.477954 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.477931    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dg2d\" (UniqueName: \"kubernetes.io/projected/4d0bd32d-3d66-4dbd-be05-66d52a429d7d-kube-api-access-5dg2d\") pod \"payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq\" (UID: \"4d0bd32d-3d66-4dbd-be05-66d52a429d7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"
Aug 05 18:14:01.593868 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.593779    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"
Aug 05 18:14:01.631302 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.631268    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"
Aug 05 18:14:01.748665 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.748616    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"]
Aug 05 18:14:01.749261 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:14:01.749230    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d0bd32d_3d66_4dbd_be05_66d52a429d7d.slice/crio-aabe6abeb90bbe69f89b0e1b951e4f3f0dcf97a425cb4497f3574f0aa17fd1e2 WatchSource:0}: Error finding container aabe6abeb90bbe69f89b0e1b951e4f3f0dcf97a425cb4497f3574f0aa17fd1e2: Status 404 returned error can't find the container with id aabe6abeb90bbe69f89b0e1b951e4f3f0dcf97a425cb4497f3574f0aa17fd1e2
Aug 05 18:14:01.853996 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.853914    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq" event={"ID":"4d0bd32d-3d66-4dbd-be05-66d52a429d7d","Type":"ContainerStarted","Data":"baa8bdd470f96868562826bd704847c0f4a0b64562dc01bcd036a15250bf3360"}
Aug 05 18:14:01.853996 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.853966    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq" event={"ID":"4d0bd32d-3d66-4dbd-be05-66d52a429d7d","Type":"ContainerStarted","Data":"aabe6abeb90bbe69f89b0e1b951e4f3f0dcf97a425cb4497f3574f0aa17fd1e2"}
Aug 05 18:14:01.854193 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.854020    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"
Aug 05 18:14:01.869112 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.869067    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq" podStartSLOduration=0.869049153 podStartE2EDuration="869.049153ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:14:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:14:01.868603521 +0000 UTC m=+3255.286918281" watchObservedRunningTime="2026-08-05 18:14:01.869049153 +0000 UTC m=+3255.287363911"
Aug 05 18:14:01.995641 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:01.995610    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"]
Aug 05 18:14:01.996668 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:14:01.996638    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d53188_1f55_4fb5_ab85_5a51a4b1ad2e.slice/crio-667e5a302d22649a569616cad2c10318c1b0cf6f2ceb349e38a0022eada27537 WatchSource:0}: Error finding container 667e5a302d22649a569616cad2c10318c1b0cf6f2ceb349e38a0022eada27537: Status 404 returned error can't find the container with id 667e5a302d22649a569616cad2c10318c1b0cf6f2ceb349e38a0022eada27537
Aug 05 18:14:02.860591 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:02.860551    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv" event={"ID":"f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e","Type":"ContainerStarted","Data":"666e20db635676666bec53514653209402666a8193e3e00c36984b9eca79749c"}
Aug 05 18:14:02.861064 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:02.860609    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv" event={"ID":"f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e","Type":"ContainerStarted","Data":"667e5a302d22649a569616cad2c10318c1b0cf6f2ceb349e38a0022eada27537"}
Aug 05 18:14:02.861064 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:02.860638    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"
Aug 05 18:14:02.877367 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:02.877324    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv" podStartSLOduration=1.877309068 podStartE2EDuration="1.877309068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:14:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:14:02.876439956 +0000 UTC m=+3256.294754714" watchObservedRunningTime="2026-08-05 18:14:02.877309068 +0000 UTC m=+3256.295623825"
Aug 05 18:14:12.863930 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:12.863900    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"
Aug 05 18:14:13.867312 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:13.867284    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"
Aug 05 18:14:30.365828 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.365795    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"]
Aug 05 18:14:30.369343 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.369318    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"
Aug 05 18:14:30.372515 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.372489    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-a83758\""
Aug 05 18:14:30.372820 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.372797    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-a83758-dockercfg-9zq2p\""
Aug 05 18:14:30.389440 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.389408    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"]
Aug 05 18:14:30.417665 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.417640    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"]
Aug 05 18:14:30.420995 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.420980    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"
Aug 05 18:14:30.439954 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.439926    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"]
Aug 05 18:14:30.504828 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.504793    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtdp8\" (UniqueName: \"kubernetes.io/projected/b64207e7-5f7f-4e0d-9528-930eb868ce0f-kube-api-access-gtdp8\") pod \"payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8\" (UID: \"b64207e7-5f7f-4e0d-9528-930eb868ce0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"
Aug 05 18:14:30.504980 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.504834    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w67dw\" (UniqueName: \"kubernetes.io/projected/0289f52e-9c7f-401a-8b3f-4643af4ed035-kube-api-access-w67dw\") pod \"payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs\" (UID: \"0289f52e-9c7f-401a-8b3f-4643af4ed035\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"
Aug 05 18:14:30.504980 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.504891    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0289f52e-9c7f-401a-8b3f-4643af4ed035-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs\" (UID: \"0289f52e-9c7f-401a-8b3f-4643af4ed035\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"
Aug 05 18:14:30.504980 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.504941    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b64207e7-5f7f-4e0d-9528-930eb868ce0f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8\" (UID: \"b64207e7-5f7f-4e0d-9528-930eb868ce0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"
Aug 05 18:14:30.606092 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.606055    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtdp8\" (UniqueName: \"kubernetes.io/projected/b64207e7-5f7f-4e0d-9528-930eb868ce0f-kube-api-access-gtdp8\") pod \"payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8\" (UID: \"b64207e7-5f7f-4e0d-9528-930eb868ce0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"
Aug 05 18:14:30.606288 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.606099    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w67dw\" (UniqueName: \"kubernetes.io/projected/0289f52e-9c7f-401a-8b3f-4643af4ed035-kube-api-access-w67dw\") pod \"payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs\" (UID: \"0289f52e-9c7f-401a-8b3f-4643af4ed035\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"
Aug 05 18:14:30.606288 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.606139    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0289f52e-9c7f-401a-8b3f-4643af4ed035-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs\" (UID: \"0289f52e-9c7f-401a-8b3f-4643af4ed035\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"
Aug 05 18:14:30.606288 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.606209    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b64207e7-5f7f-4e0d-9528-930eb868ce0f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8\" (UID: \"b64207e7-5f7f-4e0d-9528-930eb868ce0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"
Aug 05 18:14:30.606916 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.606897    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0289f52e-9c7f-401a-8b3f-4643af4ed035-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs\" (UID: \"0289f52e-9c7f-401a-8b3f-4643af4ed035\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"
Aug 05 18:14:30.606955 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.606942    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b64207e7-5f7f-4e0d-9528-930eb868ce0f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8\" (UID: \"b64207e7-5f7f-4e0d-9528-930eb868ce0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"
Aug 05 18:14:30.614274 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.614243    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtdp8\" (UniqueName: \"kubernetes.io/projected/b64207e7-5f7f-4e0d-9528-930eb868ce0f-kube-api-access-gtdp8\") pod \"payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8\" (UID: \"b64207e7-5f7f-4e0d-9528-930eb868ce0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"
Aug 05 18:14:30.614389 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.614290    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w67dw\" (UniqueName: \"kubernetes.io/projected/0289f52e-9c7f-401a-8b3f-4643af4ed035-kube-api-access-w67dw\") pod \"payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs\" (UID: \"0289f52e-9c7f-401a-8b3f-4643af4ed035\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"
Aug 05 18:14:30.680410 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.680350    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"
Aug 05 18:14:30.731833 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.731799    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"
Aug 05 18:14:30.840031 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.839999    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"]
Aug 05 18:14:30.841719 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:14:30.841677    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0289f52e_9c7f_401a_8b3f_4643af4ed035.slice/crio-3b8d68e6f0c6bd84c7c2916a21737aa56520226716df185c71198577ff199476 WatchSource:0}: Error finding container 3b8d68e6f0c6bd84c7c2916a21737aa56520226716df185c71198577ff199476: Status 404 returned error can't find the container with id 3b8d68e6f0c6bd84c7c2916a21737aa56520226716df185c71198577ff199476
Aug 05 18:14:30.888997 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.888968    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"]
Aug 05 18:14:30.890853 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:14:30.890825    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb64207e7_5f7f_4e0d_9528_930eb868ce0f.slice/crio-576221b4c6f2cf8deb38547447c4ae7eb0d514b63c4373f3a48f68b08e6b5d3e WatchSource:0}: Error finding container 576221b4c6f2cf8deb38547447c4ae7eb0d514b63c4373f3a48f68b08e6b5d3e: Status 404 returned error can't find the container with id 576221b4c6f2cf8deb38547447c4ae7eb0d514b63c4373f3a48f68b08e6b5d3e
Aug 05 18:14:30.995063 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.995038    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs" event={"ID":"0289f52e-9c7f-401a-8b3f-4643af4ed035","Type":"ContainerStarted","Data":"ac5cc79b92ad04e2f52988331cd35ac36f89b5699a5dc096b86af6a50fb31b8d"}
Aug 05 18:14:30.995203 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.995087    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs" event={"ID":"0289f52e-9c7f-401a-8b3f-4643af4ed035","Type":"ContainerStarted","Data":"3b8d68e6f0c6bd84c7c2916a21737aa56520226716df185c71198577ff199476"}
Aug 05 18:14:30.995203 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.995136    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"
Aug 05 18:14:30.996616 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.996596    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8" event={"ID":"b64207e7-5f7f-4e0d-9528-930eb868ce0f","Type":"ContainerStarted","Data":"dcd2215c6eb34983df7b1e7a20c2ec54583873e64b7409126da6450511b21196"}
Aug 05 18:14:30.996714 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.996633    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8" event={"ID":"b64207e7-5f7f-4e0d-9528-930eb868ce0f","Type":"ContainerStarted","Data":"576221b4c6f2cf8deb38547447c4ae7eb0d514b63c4373f3a48f68b08e6b5d3e"}
Aug 05 18:14:30.996714 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:30.996653    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"
Aug 05 18:14:31.013773 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:31.013658    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs" podStartSLOduration=1.013641329 podStartE2EDuration="1.013641329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:14:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:14:31.012417749 +0000 UTC m=+3284.430732507" watchObservedRunningTime="2026-08-05 18:14:31.013641329 +0000 UTC m=+3284.431956084"
Aug 05 18:14:42.003080 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:42.003052    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"
Aug 05 18:14:42.003529 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:42.003119    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"
Aug 05 18:14:42.021214 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:14:42.021167    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8" podStartSLOduration=12.021137283 podStartE2EDuration="12.021137283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:14:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:14:31.026562164 +0000 UTC m=+3284.444876922" watchObservedRunningTime="2026-08-05 18:14:42.021137283 +0000 UTC m=+3295.439452039"
Aug 05 18:15:00.420297 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:00.420265    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w7ssw_522e9cc8-a0a6-45f6-896a-1b893541b01e/payload-processing/0.log"
Aug 05 18:15:00.540569 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:00.540538    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs_0289f52e-9c7f-401a-8b3f-4643af4ed035/payload-processing/0.log"
Aug 05 18:15:50.393900 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:50.393858    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv_f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e/payload-processing/0.log"
Aug 05 18:15:50.516424 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:50.516394    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w7ssw_522e9cc8-a0a6-45f6-896a-1b893541b01e/payload-processing/0.log"
Aug 05 18:15:50.631606 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:50.631578    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs_0289f52e-9c7f-401a-8b3f-4643af4ed035/payload-processing/0.log"
Aug 05 18:15:55.018372 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.018334    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"]
Aug 05 18:15:55.022731 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.022708    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:15:55.025218 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.025198    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-fb84d0-serving-cert\""
Aug 05 18:15:55.031756 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.030938    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"]
Aug 05 18:15:55.091373 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.091349    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c37e21f9-3cad-4088-8537-d336b7205930-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq\" (UID: \"c37e21f9-3cad-4088-8537-d336b7205930\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:15:55.091521 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.091398    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb4zw\" (UniqueName: \"kubernetes.io/projected/c37e21f9-3cad-4088-8537-d336b7205930-kube-api-access-jb4zw\") pod \"maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq\" (UID: \"c37e21f9-3cad-4088-8537-d336b7205930\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:15:55.091521 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.091426    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c37e21f9-3cad-4088-8537-d336b7205930-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq\" (UID: \"c37e21f9-3cad-4088-8537-d336b7205930\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:15:55.192516 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.192481    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c37e21f9-3cad-4088-8537-d336b7205930-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq\" (UID: \"c37e21f9-3cad-4088-8537-d336b7205930\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:15:55.192692 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.192584    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c37e21f9-3cad-4088-8537-d336b7205930-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq\" (UID: \"c37e21f9-3cad-4088-8537-d336b7205930\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:15:55.192692 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.192613    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb4zw\" (UniqueName: \"kubernetes.io/projected/c37e21f9-3cad-4088-8537-d336b7205930-kube-api-access-jb4zw\") pod \"maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq\" (UID: \"c37e21f9-3cad-4088-8537-d336b7205930\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:15:55.192821 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:15:55.192727    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-serving-cert: secret "maas-api-e2e-ipp-cleanup-fb84d0-serving-cert" not found
Aug 05 18:15:55.192821 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:15:55.192804    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c37e21f9-3cad-4088-8537-d336b7205930-maas-api-tls podName:c37e21f9-3cad-4088-8537-d336b7205930 nodeName:}" failed. No retries permitted until 2026-08-05 18:15:55.692784208 +0000 UTC m=+3369.111098957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c37e21f9-3cad-4088-8537-d336b7205930-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq" (UID: "c37e21f9-3cad-4088-8537-d336b7205930") : secret "maas-api-e2e-ipp-cleanup-fb84d0-serving-cert" not found
Aug 05 18:15:55.193260 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.193238    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c37e21f9-3cad-4088-8537-d336b7205930-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq\" (UID: \"c37e21f9-3cad-4088-8537-d336b7205930\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:15:55.201338 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.201313    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb4zw\" (UniqueName: \"kubernetes.io/projected/c37e21f9-3cad-4088-8537-d336b7205930-kube-api-access-jb4zw\") pod \"maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq\" (UID: \"c37e21f9-3cad-4088-8537-d336b7205930\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:15:55.696401 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.696366    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c37e21f9-3cad-4088-8537-d336b7205930-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq\" (UID: \"c37e21f9-3cad-4088-8537-d336b7205930\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:15:55.698725 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.698701    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c37e21f9-3cad-4088-8537-d336b7205930-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq\" (UID: \"c37e21f9-3cad-4088-8537-d336b7205930\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:15:55.763659 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.763630    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"]
Aug 05 18:15:55.767872 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.767849    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"
Aug 05 18:15:55.770752 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.770726    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-fb84d0\""
Aug 05 18:15:55.770898 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.770730    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-fb84d0-dockercfg-rxzvd\""
Aug 05 18:15:55.775975 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.775949    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"]
Aug 05 18:15:55.797194 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.797138    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbzql\" (UniqueName: \"kubernetes.io/projected/a4dde33d-914d-4c61-a3b7-78897859fe6b-kube-api-access-cbzql\") pod \"payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs\" (UID: \"a4dde33d-914d-4c61-a3b7-78897859fe6b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"
Aug 05 18:15:55.797308 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.797195    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4dde33d-914d-4c61-a3b7-78897859fe6b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs\" (UID: \"a4dde33d-914d-4c61-a3b7-78897859fe6b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"
Aug 05 18:15:55.898045 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.898008    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbzql\" (UniqueName: \"kubernetes.io/projected/a4dde33d-914d-4c61-a3b7-78897859fe6b-kube-api-access-cbzql\") pod \"payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs\" (UID: \"a4dde33d-914d-4c61-a3b7-78897859fe6b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"
Aug 05 18:15:55.898238 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.898058    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4dde33d-914d-4c61-a3b7-78897859fe6b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs\" (UID: \"a4dde33d-914d-4c61-a3b7-78897859fe6b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"
Aug 05 18:15:55.898750 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.898731    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4dde33d-914d-4c61-a3b7-78897859fe6b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs\" (UID: \"a4dde33d-914d-4c61-a3b7-78897859fe6b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"
Aug 05 18:15:55.907387 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.907352    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbzql\" (UniqueName: \"kubernetes.io/projected/a4dde33d-914d-4c61-a3b7-78897859fe6b-kube-api-access-cbzql\") pod \"payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs\" (UID: \"a4dde33d-914d-4c61-a3b7-78897859fe6b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"
Aug 05 18:15:55.935107 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:55.935076    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:15:56.074209 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:56.074178    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"]
Aug 05 18:15:56.075354 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:15:56.075329    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc37e21f9_3cad_4088_8537_d336b7205930.slice/crio-0f8a780d03949c432735feac25c078f017c6dcf49509dd9a0a79d4d8b9e23efd WatchSource:0}: Error finding container 0f8a780d03949c432735feac25c078f017c6dcf49509dd9a0a79d4d8b9e23efd: Status 404 returned error can't find the container with id 0f8a780d03949c432735feac25c078f017c6dcf49509dd9a0a79d4d8b9e23efd
Aug 05 18:15:56.077178 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:56.077146    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 18:15:56.081887 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:56.081863    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"
Aug 05 18:15:56.232333 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:56.232292    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"]
Aug 05 18:15:56.239018 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:15:56.238984    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4dde33d_914d_4c61_a3b7_78897859fe6b.slice/crio-d74112d4c5166b87cdc91b6bebef621b34f9e22f0e83a1bae4cf13a06fc84789 WatchSource:0}: Error finding container d74112d4c5166b87cdc91b6bebef621b34f9e22f0e83a1bae4cf13a06fc84789: Status 404 returned error can't find the container with id d74112d4c5166b87cdc91b6bebef621b34f9e22f0e83a1bae4cf13a06fc84789
Aug 05 18:15:56.430812 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:56.430777    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs" event={"ID":"a4dde33d-914d-4c61-a3b7-78897859fe6b","Type":"ContainerStarted","Data":"977aacee641a324e5bd2be27542e1f704007717f16f482b4f2996f13c077013b"}
Aug 05 18:15:56.431211 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:56.431146    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs" event={"ID":"a4dde33d-914d-4c61-a3b7-78897859fe6b","Type":"ContainerStarted","Data":"d74112d4c5166b87cdc91b6bebef621b34f9e22f0e83a1bae4cf13a06fc84789"}
Aug 05 18:15:56.431343 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:56.431219    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"
Aug 05 18:15:56.432257 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:56.432235    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq" event={"ID":"c37e21f9-3cad-4088-8537-d336b7205930","Type":"ContainerStarted","Data":"0f8a780d03949c432735feac25c078f017c6dcf49509dd9a0a79d4d8b9e23efd"}
Aug 05 18:15:56.449134 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:56.449095    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs" podStartSLOduration=1.449078826 podStartE2EDuration="1.449078826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:15:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:15:56.448946726 +0000 UTC m=+3369.867261511" watchObservedRunningTime="2026-08-05 18:15:56.449078826 +0000 UTC m=+3369.867393586"
Aug 05 18:15:57.439217 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:57.439185    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq" event={"ID":"c37e21f9-3cad-4088-8537-d336b7205930","Type":"ContainerStarted","Data":"14b45e8aa628735a363348b5ff86e7fcae8ef177c775b86276327a859e31a6b3"}
Aug 05 18:15:57.439709 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:57.439248    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:15:57.455272 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:15:57.455225    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq" podStartSLOduration=2.8592486790000002 podStartE2EDuration="3.455210274s" totalImagesPullingTime="595.961595ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:15:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:15:57.454624586 +0000 UTC m=+3370.872939364" watchObservedRunningTime="2026-08-05 18:15:57.455210274 +0000 UTC m=+3370.873525031"
Aug 05 18:16:03.449700 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:03.449672    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:16:07.441027 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:07.440989    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"
Aug 05 18:16:25.165095 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.165057    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"]
Aug 05 18:16:25.165570 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.165341    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq" podUID="c37e21f9-3cad-4088-8537-d336b7205930" containerName="maas-api" containerID="cri-o://14b45e8aa628735a363348b5ff86e7fcae8ef177c775b86276327a859e31a6b3" gracePeriod=30
Aug 05 18:16:25.216973 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.216930    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"]
Aug 05 18:16:25.217217 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.217189    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs" podUID="a4dde33d-914d-4c61-a3b7-78897859fe6b" containerName="payload-processing" containerID="cri-o://977aacee641a324e5bd2be27542e1f704007717f16f482b4f2996f13c077013b" gracePeriod=30
Aug 05 18:16:25.482326 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.482297    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:16:25.512525 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.512283    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"
Aug 05 18:16:25.555260 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.555233    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c37e21f9-3cad-4088-8537-d336b7205930-maas-api-tls\") pod \"c37e21f9-3cad-4088-8537-d336b7205930\" (UID: \"c37e21f9-3cad-4088-8537-d336b7205930\") "
Aug 05 18:16:25.555398 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.555283    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb4zw\" (UniqueName: \"kubernetes.io/projected/c37e21f9-3cad-4088-8537-d336b7205930-kube-api-access-jb4zw\") pod \"c37e21f9-3cad-4088-8537-d336b7205930\" (UID: \"c37e21f9-3cad-4088-8537-d336b7205930\") "
Aug 05 18:16:25.555461 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.555435    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c37e21f9-3cad-4088-8537-d336b7205930-openshift-service-ca\") pod \"c37e21f9-3cad-4088-8537-d336b7205930\" (UID: \"c37e21f9-3cad-4088-8537-d336b7205930\") "
Aug 05 18:16:25.555763 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.555731    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c37e21f9-3cad-4088-8537-d336b7205930-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c37e21f9-3cad-4088-8537-d336b7205930" (UID: "c37e21f9-3cad-4088-8537-d336b7205930"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:16:25.557304 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.557281    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c37e21f9-3cad-4088-8537-d336b7205930-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c37e21f9-3cad-4088-8537-d336b7205930" (UID: "c37e21f9-3cad-4088-8537-d336b7205930"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:16:25.557407 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.557387    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c37e21f9-3cad-4088-8537-d336b7205930-kube-api-access-jb4zw" (OuterVolumeSpecName: "kube-api-access-jb4zw") pod "c37e21f9-3cad-4088-8537-d336b7205930" (UID: "c37e21f9-3cad-4088-8537-d336b7205930"). InnerVolumeSpecName "kube-api-access-jb4zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:16:25.582357 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.582329    2556 generic.go:356] "Generic (PLEG): container finished" podID="a4dde33d-914d-4c61-a3b7-78897859fe6b" containerID="977aacee641a324e5bd2be27542e1f704007717f16f482b4f2996f13c077013b" exitCode=0
Aug 05 18:16:25.582487 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.582375    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"
Aug 05 18:16:25.582487 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.582441    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs" event={"ID":"a4dde33d-914d-4c61-a3b7-78897859fe6b","Type":"ContainerDied","Data":"977aacee641a324e5bd2be27542e1f704007717f16f482b4f2996f13c077013b"}
Aug 05 18:16:25.582599 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.582501    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs" event={"ID":"a4dde33d-914d-4c61-a3b7-78897859fe6b","Type":"ContainerDied","Data":"d74112d4c5166b87cdc91b6bebef621b34f9e22f0e83a1bae4cf13a06fc84789"}
Aug 05 18:16:25.582599 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.582566    2556 scope.go:117] "RemoveContainer" containerID="977aacee641a324e5bd2be27542e1f704007717f16f482b4f2996f13c077013b"
Aug 05 18:16:25.583837 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.583819    2556 generic.go:356] "Generic (PLEG): container finished" podID="c37e21f9-3cad-4088-8537-d336b7205930" containerID="14b45e8aa628735a363348b5ff86e7fcae8ef177c775b86276327a859e31a6b3" exitCode=0
Aug 05 18:16:25.583912 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.583862    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"
Aug 05 18:16:25.583950 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.583913    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq" event={"ID":"c37e21f9-3cad-4088-8537-d336b7205930","Type":"ContainerDied","Data":"14b45e8aa628735a363348b5ff86e7fcae8ef177c775b86276327a859e31a6b3"}
Aug 05 18:16:25.583994 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.583963    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq" event={"ID":"c37e21f9-3cad-4088-8537-d336b7205930","Type":"ContainerDied","Data":"0f8a780d03949c432735feac25c078f017c6dcf49509dd9a0a79d4d8b9e23efd"}
Aug 05 18:16:25.591899 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.591882    2556 scope.go:117] "RemoveContainer" containerID="977aacee641a324e5bd2be27542e1f704007717f16f482b4f2996f13c077013b"
Aug 05 18:16:25.592185 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:16:25.592141    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"977aacee641a324e5bd2be27542e1f704007717f16f482b4f2996f13c077013b\": container with ID starting with 977aacee641a324e5bd2be27542e1f704007717f16f482b4f2996f13c077013b not found: ID does not exist" containerID="977aacee641a324e5bd2be27542e1f704007717f16f482b4f2996f13c077013b"
Aug 05 18:16:25.592303 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.592189    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"977aacee641a324e5bd2be27542e1f704007717f16f482b4f2996f13c077013b"} err="failed to get container status \"977aacee641a324e5bd2be27542e1f704007717f16f482b4f2996f13c077013b\": rpc error: code = NotFound desc = could not find container \"977aacee641a324e5bd2be27542e1f704007717f16f482b4f2996f13c077013b\": container with ID starting with 977aacee641a324e5bd2be27542e1f704007717f16f482b4f2996f13c077013b not found: ID does not exist"
Aug 05 18:16:25.592303 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.592211    2556 scope.go:117] "RemoveContainer" containerID="14b45e8aa628735a363348b5ff86e7fcae8ef177c775b86276327a859e31a6b3"
Aug 05 18:16:25.600231 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.600215    2556 scope.go:117] "RemoveContainer" containerID="14b45e8aa628735a363348b5ff86e7fcae8ef177c775b86276327a859e31a6b3"
Aug 05 18:16:25.600463 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:16:25.600446    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14b45e8aa628735a363348b5ff86e7fcae8ef177c775b86276327a859e31a6b3\": container with ID starting with 14b45e8aa628735a363348b5ff86e7fcae8ef177c775b86276327a859e31a6b3 not found: ID does not exist" containerID="14b45e8aa628735a363348b5ff86e7fcae8ef177c775b86276327a859e31a6b3"
Aug 05 18:16:25.600520 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.600469    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14b45e8aa628735a363348b5ff86e7fcae8ef177c775b86276327a859e31a6b3"} err="failed to get container status \"14b45e8aa628735a363348b5ff86e7fcae8ef177c775b86276327a859e31a6b3\": rpc error: code = NotFound desc = could not find container \"14b45e8aa628735a363348b5ff86e7fcae8ef177c775b86276327a859e31a6b3\": container with ID starting with 14b45e8aa628735a363348b5ff86e7fcae8ef177c775b86276327a859e31a6b3 not found: ID does not exist"
Aug 05 18:16:25.605094 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.605076    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"]
Aug 05 18:16:25.608759 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.608738    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-fb84d0-85dff66dc6-b6kcq"]
Aug 05 18:16:25.656336 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.656313    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbzql\" (UniqueName: \"kubernetes.io/projected/a4dde33d-914d-4c61-a3b7-78897859fe6b-kube-api-access-cbzql\") pod \"a4dde33d-914d-4c61-a3b7-78897859fe6b\" (UID: \"a4dde33d-914d-4c61-a3b7-78897859fe6b\") "
Aug 05 18:16:25.656442 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.656383    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4dde33d-914d-4c61-a3b7-78897859fe6b-plugins-config-volume\") pod \"a4dde33d-914d-4c61-a3b7-78897859fe6b\" (UID: \"a4dde33d-914d-4c61-a3b7-78897859fe6b\") "
Aug 05 18:16:25.656718 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.656702    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c37e21f9-3cad-4088-8537-d336b7205930-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:16:25.656780 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.656722    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb4zw\" (UniqueName: \"kubernetes.io/projected/c37e21f9-3cad-4088-8537-d336b7205930-kube-api-access-jb4zw\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:16:25.656780 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.656736    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c37e21f9-3cad-4088-8537-d336b7205930-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:16:25.656780 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.656761    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4dde33d-914d-4c61-a3b7-78897859fe6b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a4dde33d-914d-4c61-a3b7-78897859fe6b" (UID: "a4dde33d-914d-4c61-a3b7-78897859fe6b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:16:25.658235 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.658210    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4dde33d-914d-4c61-a3b7-78897859fe6b-kube-api-access-cbzql" (OuterVolumeSpecName: "kube-api-access-cbzql") pod "a4dde33d-914d-4c61-a3b7-78897859fe6b" (UID: "a4dde33d-914d-4c61-a3b7-78897859fe6b"). InnerVolumeSpecName "kube-api-access-cbzql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:16:25.757992 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.757970    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbzql\" (UniqueName: \"kubernetes.io/projected/a4dde33d-914d-4c61-a3b7-78897859fe6b-kube-api-access-cbzql\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:16:25.757992 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.757991    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4dde33d-914d-4c61-a3b7-78897859fe6b-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:16:25.918609 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.918574    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"]
Aug 05 18:16:25.940545 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:25.940520    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-fb84d0-5dc44d44fb-6frhs"]
Aug 05 18:16:27.097897 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:27.097863    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4dde33d-914d-4c61-a3b7-78897859fe6b" path="/var/lib/kubelet/pods/a4dde33d-914d-4c61-a3b7-78897859fe6b/volumes"
Aug 05 18:16:27.098307 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:27.098273    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c37e21f9-3cad-4088-8537-d336b7205930" path="/var/lib/kubelet/pods/c37e21f9-3cad-4088-8537-d336b7205930/volumes"
Aug 05 18:16:51.245430 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.245368    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"]
Aug 05 18:16:51.245939 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.245681    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv" podUID="f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e" containerName="payload-processing" containerID="cri-o://666e20db635676666bec53514653209402666a8193e3e00c36984b9eca79749c" gracePeriod=30
Aug 05 18:16:51.250294 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.250072    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"]
Aug 05 18:16:51.250518 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.250490    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq" podUID="4d0bd32d-3d66-4dbd-be05-66d52a429d7d" containerName="payload-pre-processing" containerID="cri-o://baa8bdd470f96868562826bd704847c0f4a0b64562dc01bcd036a15250bf3360" gracePeriod=30
Aug 05 18:16:51.540211 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.540184    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"
Aug 05 18:16:51.542955 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.542940    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"
Aug 05 18:16:51.571427 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.571205    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95zwr\" (UniqueName: \"kubernetes.io/projected/f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e-kube-api-access-95zwr\") pod \"f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e\" (UID: \"f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e\") "
Aug 05 18:16:51.571427 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.571278    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e-plugins-config-volume\") pod \"f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e\" (UID: \"f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e\") "
Aug 05 18:16:51.571427 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.571363    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d0bd32d-3d66-4dbd-be05-66d52a429d7d-plugins-config-volume\") pod \"4d0bd32d-3d66-4dbd-be05-66d52a429d7d\" (UID: \"4d0bd32d-3d66-4dbd-be05-66d52a429d7d\") "
Aug 05 18:16:51.571427 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.571397    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dg2d\" (UniqueName: \"kubernetes.io/projected/4d0bd32d-3d66-4dbd-be05-66d52a429d7d-kube-api-access-5dg2d\") pod \"4d0bd32d-3d66-4dbd-be05-66d52a429d7d\" (UID: \"4d0bd32d-3d66-4dbd-be05-66d52a429d7d\") "
Aug 05 18:16:51.571973 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.571919    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e" (UID: "f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:16:51.572190 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.572135    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d0bd32d-3d66-4dbd-be05-66d52a429d7d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4d0bd32d-3d66-4dbd-be05-66d52a429d7d" (UID: "4d0bd32d-3d66-4dbd-be05-66d52a429d7d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:16:51.574037 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.573999    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e-kube-api-access-95zwr" (OuterVolumeSpecName: "kube-api-access-95zwr") pod "f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e" (UID: "f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e"). InnerVolumeSpecName "kube-api-access-95zwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:16:51.574300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.574265    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d0bd32d-3d66-4dbd-be05-66d52a429d7d-kube-api-access-5dg2d" (OuterVolumeSpecName: "kube-api-access-5dg2d") pod "4d0bd32d-3d66-4dbd-be05-66d52a429d7d" (UID: "4d0bd32d-3d66-4dbd-be05-66d52a429d7d"). InnerVolumeSpecName "kube-api-access-5dg2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:16:51.672512 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.672478    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d0bd32d-3d66-4dbd-be05-66d52a429d7d-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:16:51.672512 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.672507    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dg2d\" (UniqueName: \"kubernetes.io/projected/4d0bd32d-3d66-4dbd-be05-66d52a429d7d-kube-api-access-5dg2d\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:16:51.672512 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.672518    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95zwr\" (UniqueName: \"kubernetes.io/projected/f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e-kube-api-access-95zwr\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:16:51.672741 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.672526    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:16:51.724803 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.724769    2556 generic.go:356] "Generic (PLEG): container finished" podID="f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e" containerID="666e20db635676666bec53514653209402666a8193e3e00c36984b9eca79749c" exitCode=0
Aug 05 18:16:51.724955 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.724815    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"
Aug 05 18:16:51.724955 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.724858    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv" event={"ID":"f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e","Type":"ContainerDied","Data":"666e20db635676666bec53514653209402666a8193e3e00c36984b9eca79749c"}
Aug 05 18:16:51.724955 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.724906    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv" event={"ID":"f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e","Type":"ContainerDied","Data":"667e5a302d22649a569616cad2c10318c1b0cf6f2ceb349e38a0022eada27537"}
Aug 05 18:16:51.724955 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.724928    2556 scope.go:117] "RemoveContainer" containerID="666e20db635676666bec53514653209402666a8193e3e00c36984b9eca79749c"
Aug 05 18:16:51.726243 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.726214    2556 generic.go:356] "Generic (PLEG): container finished" podID="4d0bd32d-3d66-4dbd-be05-66d52a429d7d" containerID="baa8bdd470f96868562826bd704847c0f4a0b64562dc01bcd036a15250bf3360" exitCode=0
Aug 05 18:16:51.726365 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.726257    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"
Aug 05 18:16:51.726365 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.726296    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq" event={"ID":"4d0bd32d-3d66-4dbd-be05-66d52a429d7d","Type":"ContainerDied","Data":"baa8bdd470f96868562826bd704847c0f4a0b64562dc01bcd036a15250bf3360"}
Aug 05 18:16:51.726365 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.726356    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq" event={"ID":"4d0bd32d-3d66-4dbd-be05-66d52a429d7d","Type":"ContainerDied","Data":"aabe6abeb90bbe69f89b0e1b951e4f3f0dcf97a425cb4497f3574f0aa17fd1e2"}
Aug 05 18:16:51.735015 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.734822    2556 scope.go:117] "RemoveContainer" containerID="666e20db635676666bec53514653209402666a8193e3e00c36984b9eca79749c"
Aug 05 18:16:51.735209 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:16:51.735191    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"666e20db635676666bec53514653209402666a8193e3e00c36984b9eca79749c\": container with ID starting with 666e20db635676666bec53514653209402666a8193e3e00c36984b9eca79749c not found: ID does not exist" containerID="666e20db635676666bec53514653209402666a8193e3e00c36984b9eca79749c"
Aug 05 18:16:51.735317 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.735221    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"666e20db635676666bec53514653209402666a8193e3e00c36984b9eca79749c"} err="failed to get container status \"666e20db635676666bec53514653209402666a8193e3e00c36984b9eca79749c\": rpc error: code = NotFound desc = could not find container \"666e20db635676666bec53514653209402666a8193e3e00c36984b9eca79749c\": container with ID starting with 666e20db635676666bec53514653209402666a8193e3e00c36984b9eca79749c not found: ID does not exist"
Aug 05 18:16:51.735317 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.735245    2556 scope.go:117] "RemoveContainer" containerID="baa8bdd470f96868562826bd704847c0f4a0b64562dc01bcd036a15250bf3360"
Aug 05 18:16:51.743180 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.743144    2556 scope.go:117] "RemoveContainer" containerID="baa8bdd470f96868562826bd704847c0f4a0b64562dc01bcd036a15250bf3360"
Aug 05 18:16:51.743413 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:16:51.743394    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baa8bdd470f96868562826bd704847c0f4a0b64562dc01bcd036a15250bf3360\": container with ID starting with baa8bdd470f96868562826bd704847c0f4a0b64562dc01bcd036a15250bf3360 not found: ID does not exist" containerID="baa8bdd470f96868562826bd704847c0f4a0b64562dc01bcd036a15250bf3360"
Aug 05 18:16:51.743463 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.743420    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baa8bdd470f96868562826bd704847c0f4a0b64562dc01bcd036a15250bf3360"} err="failed to get container status \"baa8bdd470f96868562826bd704847c0f4a0b64562dc01bcd036a15250bf3360\": rpc error: code = NotFound desc = could not find container \"baa8bdd470f96868562826bd704847c0f4a0b64562dc01bcd036a15250bf3360\": container with ID starting with baa8bdd470f96868562826bd704847c0f4a0b64562dc01bcd036a15250bf3360 not found: ID does not exist"
Aug 05 18:16:51.749719 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.749698    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"]
Aug 05 18:16:51.754118 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.754096    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4babb1-695b4f5d7b-ssphq"]
Aug 05 18:16:51.762976 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.762957    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"]
Aug 05 18:16:51.768646 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:51.768602    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4babb1-665c94dc46-h4ngv"]
Aug 05 18:16:53.097039 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:53.096938    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d0bd32d-3d66-4dbd-be05-66d52a429d7d" path="/var/lib/kubelet/pods/4d0bd32d-3d66-4dbd-be05-66d52a429d7d/volumes"
Aug 05 18:16:53.099623 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:16:53.097368    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e" path="/var/lib/kubelet/pods/f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e/volumes"
Aug 05 18:18:13.293729 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.293650    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"]
Aug 05 18:18:13.294282 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.293898    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs" podUID="0289f52e-9c7f-401a-8b3f-4643af4ed035" containerName="payload-processing" containerID="cri-o://ac5cc79b92ad04e2f52988331cd35ac36f89b5699a5dc096b86af6a50fb31b8d" gracePeriod=30
Aug 05 18:18:13.303262 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.303232    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"]
Aug 05 18:18:13.303552 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.303492    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8" podUID="b64207e7-5f7f-4e0d-9528-930eb868ce0f" containerName="payload-pre-processing" containerID="cri-o://dcd2215c6eb34983df7b1e7a20c2ec54583873e64b7409126da6450511b21196" gracePeriod=30
Aug 05 18:18:13.554107 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.554053    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"
Aug 05 18:18:13.582863 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.582842    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"
Aug 05 18:18:13.636093 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.636049    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b64207e7-5f7f-4e0d-9528-930eb868ce0f-plugins-config-volume\") pod \"b64207e7-5f7f-4e0d-9528-930eb868ce0f\" (UID: \"b64207e7-5f7f-4e0d-9528-930eb868ce0f\") "
Aug 05 18:18:13.636244 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.636137    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtdp8\" (UniqueName: \"kubernetes.io/projected/b64207e7-5f7f-4e0d-9528-930eb868ce0f-kube-api-access-gtdp8\") pod \"b64207e7-5f7f-4e0d-9528-930eb868ce0f\" (UID: \"b64207e7-5f7f-4e0d-9528-930eb868ce0f\") "
Aug 05 18:18:13.636244 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.636233    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w67dw\" (UniqueName: \"kubernetes.io/projected/0289f52e-9c7f-401a-8b3f-4643af4ed035-kube-api-access-w67dw\") pod \"0289f52e-9c7f-401a-8b3f-4643af4ed035\" (UID: \"0289f52e-9c7f-401a-8b3f-4643af4ed035\") "
Aug 05 18:18:13.636344 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.636280    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0289f52e-9c7f-401a-8b3f-4643af4ed035-plugins-config-volume\") pod \"0289f52e-9c7f-401a-8b3f-4643af4ed035\" (UID: \"0289f52e-9c7f-401a-8b3f-4643af4ed035\") "
Aug 05 18:18:13.636482 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.636455    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b64207e7-5f7f-4e0d-9528-930eb868ce0f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b64207e7-5f7f-4e0d-9528-930eb868ce0f" (UID: "b64207e7-5f7f-4e0d-9528-930eb868ce0f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:18:13.636709 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.636681    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0289f52e-9c7f-401a-8b3f-4643af4ed035-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0289f52e-9c7f-401a-8b3f-4643af4ed035" (UID: "0289f52e-9c7f-401a-8b3f-4643af4ed035"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:18:13.638232 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.638205    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b64207e7-5f7f-4e0d-9528-930eb868ce0f-kube-api-access-gtdp8" (OuterVolumeSpecName: "kube-api-access-gtdp8") pod "b64207e7-5f7f-4e0d-9528-930eb868ce0f" (UID: "b64207e7-5f7f-4e0d-9528-930eb868ce0f"). InnerVolumeSpecName "kube-api-access-gtdp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:18:13.638319 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.638243    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0289f52e-9c7f-401a-8b3f-4643af4ed035-kube-api-access-w67dw" (OuterVolumeSpecName: "kube-api-access-w67dw") pod "0289f52e-9c7f-401a-8b3f-4643af4ed035" (UID: "0289f52e-9c7f-401a-8b3f-4643af4ed035"). InnerVolumeSpecName "kube-api-access-w67dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:18:13.736977 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.736953    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w67dw\" (UniqueName: \"kubernetes.io/projected/0289f52e-9c7f-401a-8b3f-4643af4ed035-kube-api-access-w67dw\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:13.736977 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.736976    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0289f52e-9c7f-401a-8b3f-4643af4ed035-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:13.737114 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.736985    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b64207e7-5f7f-4e0d-9528-930eb868ce0f-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:13.737114 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:13.736994    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtdp8\" (UniqueName: \"kubernetes.io/projected/b64207e7-5f7f-4e0d-9528-930eb868ce0f-kube-api-access-gtdp8\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:14.131081 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.131046    2556 generic.go:356] "Generic (PLEG): container finished" podID="0289f52e-9c7f-401a-8b3f-4643af4ed035" containerID="ac5cc79b92ad04e2f52988331cd35ac36f89b5699a5dc096b86af6a50fb31b8d" exitCode=0
Aug 05 18:18:14.131288 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.131113    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"
Aug 05 18:18:14.131288 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.131126    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs" event={"ID":"0289f52e-9c7f-401a-8b3f-4643af4ed035","Type":"ContainerDied","Data":"ac5cc79b92ad04e2f52988331cd35ac36f89b5699a5dc096b86af6a50fb31b8d"}
Aug 05 18:18:14.131288 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.131204    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs" event={"ID":"0289f52e-9c7f-401a-8b3f-4643af4ed035","Type":"ContainerDied","Data":"3b8d68e6f0c6bd84c7c2916a21737aa56520226716df185c71198577ff199476"}
Aug 05 18:18:14.131288 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.131231    2556 scope.go:117] "RemoveContainer" containerID="ac5cc79b92ad04e2f52988331cd35ac36f89b5699a5dc096b86af6a50fb31b8d"
Aug 05 18:18:14.132448 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.132421    2556 generic.go:356] "Generic (PLEG): container finished" podID="b64207e7-5f7f-4e0d-9528-930eb868ce0f" containerID="dcd2215c6eb34983df7b1e7a20c2ec54583873e64b7409126da6450511b21196" exitCode=0
Aug 05 18:18:14.132540 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.132478    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"
Aug 05 18:18:14.132540 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.132494    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8" event={"ID":"b64207e7-5f7f-4e0d-9528-930eb868ce0f","Type":"ContainerDied","Data":"dcd2215c6eb34983df7b1e7a20c2ec54583873e64b7409126da6450511b21196"}
Aug 05 18:18:14.132540 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.132525    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8" event={"ID":"b64207e7-5f7f-4e0d-9528-930eb868ce0f","Type":"ContainerDied","Data":"576221b4c6f2cf8deb38547447c4ae7eb0d514b63c4373f3a48f68b08e6b5d3e"}
Aug 05 18:18:14.140345 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.140327    2556 scope.go:117] "RemoveContainer" containerID="ac5cc79b92ad04e2f52988331cd35ac36f89b5699a5dc096b86af6a50fb31b8d"
Aug 05 18:18:14.140617 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:18:14.140590    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac5cc79b92ad04e2f52988331cd35ac36f89b5699a5dc096b86af6a50fb31b8d\": container with ID starting with ac5cc79b92ad04e2f52988331cd35ac36f89b5699a5dc096b86af6a50fb31b8d not found: ID does not exist" containerID="ac5cc79b92ad04e2f52988331cd35ac36f89b5699a5dc096b86af6a50fb31b8d"
Aug 05 18:18:14.140732 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.140626    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac5cc79b92ad04e2f52988331cd35ac36f89b5699a5dc096b86af6a50fb31b8d"} err="failed to get container status \"ac5cc79b92ad04e2f52988331cd35ac36f89b5699a5dc096b86af6a50fb31b8d\": rpc error: code = NotFound desc = could not find container \"ac5cc79b92ad04e2f52988331cd35ac36f89b5699a5dc096b86af6a50fb31b8d\": container with ID starting with ac5cc79b92ad04e2f52988331cd35ac36f89b5699a5dc096b86af6a50fb31b8d not found: ID does not exist"
Aug 05 18:18:14.140732 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.140649    2556 scope.go:117] "RemoveContainer" containerID="dcd2215c6eb34983df7b1e7a20c2ec54583873e64b7409126da6450511b21196"
Aug 05 18:18:14.150838 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.150799    2556 scope.go:117] "RemoveContainer" containerID="dcd2215c6eb34983df7b1e7a20c2ec54583873e64b7409126da6450511b21196"
Aug 05 18:18:14.151030 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:18:14.151010    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcd2215c6eb34983df7b1e7a20c2ec54583873e64b7409126da6450511b21196\": container with ID starting with dcd2215c6eb34983df7b1e7a20c2ec54583873e64b7409126da6450511b21196 not found: ID does not exist" containerID="dcd2215c6eb34983df7b1e7a20c2ec54583873e64b7409126da6450511b21196"
Aug 05 18:18:14.151077 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.151038    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcd2215c6eb34983df7b1e7a20c2ec54583873e64b7409126da6450511b21196"} err="failed to get container status \"dcd2215c6eb34983df7b1e7a20c2ec54583873e64b7409126da6450511b21196\": rpc error: code = NotFound desc = could not find container \"dcd2215c6eb34983df7b1e7a20c2ec54583873e64b7409126da6450511b21196\": container with ID starting with dcd2215c6eb34983df7b1e7a20c2ec54583873e64b7409126da6450511b21196 not found: ID does not exist"
Aug 05 18:18:14.154642 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.154621    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"]
Aug 05 18:18:14.159811 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.159792    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a83758-776f965dd4-rddbs"]
Aug 05 18:18:14.170635 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.170613    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"]
Aug 05 18:18:14.175454 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:14.175433    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a83758-546f48c6bd-kh5s8"]
Aug 05 18:18:15.097630 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:15.097596    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0289f52e-9c7f-401a-8b3f-4643af4ed035" path="/var/lib/kubelet/pods/0289f52e-9c7f-401a-8b3f-4643af4ed035/volumes"
Aug 05 18:18:15.098006 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:15.097956    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b64207e7-5f7f-4e0d-9528-930eb868ce0f" path="/var/lib/kubelet/pods/b64207e7-5f7f-4e0d-9528-930eb868ce0f/volumes"
Aug 05 18:18:30.911293 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:30.911263    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"]
Aug 05 18:18:30.911712 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:30.911447    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b" podUID="7b43900b-d530-44a1-8e77-d6b412fe2f65" containerName="maas-api" containerID="cri-o://cf3789a3e8c56bcbfdbd0f2291e0e218bd1612f7d79e67e018e816184fa9032a" gracePeriod=30
Aug 05 18:18:30.976697 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:30.976666    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"]
Aug 05 18:18:30.976940 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:30.976914    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v" podUID="384cf89a-b510-478d-9411-85a3f89d7fc7" containerName="payload-processing" containerID="cri-o://4e54b323d4098f8aabf8c3ff9e49e21226582d8fa13fd6310e897afeb43f7bdb" gracePeriod=30
Aug 05 18:18:30.996255 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:30.996133    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"]
Aug 05 18:18:30.997201 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:30.997170    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c" podUID="1f79e26a-9c6c-4347-8088-2356b2be9445" containerName="payload-pre-processing" containerID="cri-o://3e352b1c6bdafcccdf59e1932740b650eac462e381d3363abffdc86a4f9ac036" gracePeriod=30
Aug 05 18:18:31.215720 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.215682    2556 generic.go:356] "Generic (PLEG): container finished" podID="7b43900b-d530-44a1-8e77-d6b412fe2f65" containerID="cf3789a3e8c56bcbfdbd0f2291e0e218bd1612f7d79e67e018e816184fa9032a" exitCode=0
Aug 05 18:18:31.215894 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.215769    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b" event={"ID":"7b43900b-d530-44a1-8e77-d6b412fe2f65","Type":"ContainerDied","Data":"cf3789a3e8c56bcbfdbd0f2291e0e218bd1612f7d79e67e018e816184fa9032a"}
Aug 05 18:18:31.217617 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.217587    2556 generic.go:356] "Generic (PLEG): container finished" podID="1f79e26a-9c6c-4347-8088-2356b2be9445" containerID="3e352b1c6bdafcccdf59e1932740b650eac462e381d3363abffdc86a4f9ac036" exitCode=0
Aug 05 18:18:31.217830 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.217672    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c" event={"ID":"1f79e26a-9c6c-4347-8088-2356b2be9445","Type":"ContainerDied","Data":"3e352b1c6bdafcccdf59e1932740b650eac462e381d3363abffdc86a4f9ac036"}
Aug 05 18:18:31.219827 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.219797    2556 generic.go:356] "Generic (PLEG): container finished" podID="384cf89a-b510-478d-9411-85a3f89d7fc7" containerID="4e54b323d4098f8aabf8c3ff9e49e21226582d8fa13fd6310e897afeb43f7bdb" exitCode=0
Aug 05 18:18:31.220048 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.219863    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v" event={"ID":"384cf89a-b510-478d-9411-85a3f89d7fc7","Type":"ContainerDied","Data":"4e54b323d4098f8aabf8c3ff9e49e21226582d8fa13fd6310e897afeb43f7bdb"}
Aug 05 18:18:31.251034 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.251011    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"
Aug 05 18:18:31.289532 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.289508    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"
Aug 05 18:18:31.321305 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.321284    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"
Aug 05 18:18:31.382305 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.382278    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f79e26a-9c6c-4347-8088-2356b2be9445-plugins-config-volume\") pod \"1f79e26a-9c6c-4347-8088-2356b2be9445\" (UID: \"1f79e26a-9c6c-4347-8088-2356b2be9445\") "
Aug 05 18:18:31.382422 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.382322    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/384cf89a-b510-478d-9411-85a3f89d7fc7-plugins-config-volume\") pod \"384cf89a-b510-478d-9411-85a3f89d7fc7\" (UID: \"384cf89a-b510-478d-9411-85a3f89d7fc7\") "
Aug 05 18:18:31.382422 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.382350    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqm7b\" (UniqueName: \"kubernetes.io/projected/384cf89a-b510-478d-9411-85a3f89d7fc7-kube-api-access-jqm7b\") pod \"384cf89a-b510-478d-9411-85a3f89d7fc7\" (UID: \"384cf89a-b510-478d-9411-85a3f89d7fc7\") "
Aug 05 18:18:31.382422 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.382386    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b43900b-d530-44a1-8e77-d6b412fe2f65-openshift-service-ca\") pod \"7b43900b-d530-44a1-8e77-d6b412fe2f65\" (UID: \"7b43900b-d530-44a1-8e77-d6b412fe2f65\") "
Aug 05 18:18:31.382422 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.382405    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b43900b-d530-44a1-8e77-d6b412fe2f65-maas-api-tls\") pod \"7b43900b-d530-44a1-8e77-d6b412fe2f65\" (UID: \"7b43900b-d530-44a1-8e77-d6b412fe2f65\") "
Aug 05 18:18:31.382635 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.382424    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w257j\" (UniqueName: \"kubernetes.io/projected/7b43900b-d530-44a1-8e77-d6b412fe2f65-kube-api-access-w257j\") pod \"7b43900b-d530-44a1-8e77-d6b412fe2f65\" (UID: \"7b43900b-d530-44a1-8e77-d6b412fe2f65\") "
Aug 05 18:18:31.382635 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.382583    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lrql\" (UniqueName: \"kubernetes.io/projected/1f79e26a-9c6c-4347-8088-2356b2be9445-kube-api-access-6lrql\") pod \"1f79e26a-9c6c-4347-8088-2356b2be9445\" (UID: \"1f79e26a-9c6c-4347-8088-2356b2be9445\") "
Aug 05 18:18:31.382747 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.382717    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f79e26a-9c6c-4347-8088-2356b2be9445-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1f79e26a-9c6c-4347-8088-2356b2be9445" (UID: "1f79e26a-9c6c-4347-8088-2356b2be9445"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:18:31.382801 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.382782    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b43900b-d530-44a1-8e77-d6b412fe2f65-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7b43900b-d530-44a1-8e77-d6b412fe2f65" (UID: "7b43900b-d530-44a1-8e77-d6b412fe2f65"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:18:31.382870 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.382823    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/384cf89a-b510-478d-9411-85a3f89d7fc7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "384cf89a-b510-478d-9411-85a3f89d7fc7" (UID: "384cf89a-b510-478d-9411-85a3f89d7fc7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:18:31.382988 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.382970    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b43900b-d530-44a1-8e77-d6b412fe2f65-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:31.383046 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.382996    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f79e26a-9c6c-4347-8088-2356b2be9445-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:31.383046 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.383011    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/384cf89a-b510-478d-9411-85a3f89d7fc7-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:31.384595 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.384561    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b43900b-d530-44a1-8e77-d6b412fe2f65-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7b43900b-d530-44a1-8e77-d6b412fe2f65" (UID: "7b43900b-d530-44a1-8e77-d6b412fe2f65"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:18:31.384595 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.384574    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/384cf89a-b510-478d-9411-85a3f89d7fc7-kube-api-access-jqm7b" (OuterVolumeSpecName: "kube-api-access-jqm7b") pod "384cf89a-b510-478d-9411-85a3f89d7fc7" (UID: "384cf89a-b510-478d-9411-85a3f89d7fc7"). InnerVolumeSpecName "kube-api-access-jqm7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:18:31.384734 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.384597    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b43900b-d530-44a1-8e77-d6b412fe2f65-kube-api-access-w257j" (OuterVolumeSpecName: "kube-api-access-w257j") pod "7b43900b-d530-44a1-8e77-d6b412fe2f65" (UID: "7b43900b-d530-44a1-8e77-d6b412fe2f65"). InnerVolumeSpecName "kube-api-access-w257j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:18:31.384734 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.384689    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f79e26a-9c6c-4347-8088-2356b2be9445-kube-api-access-6lrql" (OuterVolumeSpecName: "kube-api-access-6lrql") pod "1f79e26a-9c6c-4347-8088-2356b2be9445" (UID: "1f79e26a-9c6c-4347-8088-2356b2be9445"). InnerVolumeSpecName "kube-api-access-6lrql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:18:31.484225 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.484198    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqm7b\" (UniqueName: \"kubernetes.io/projected/384cf89a-b510-478d-9411-85a3f89d7fc7-kube-api-access-jqm7b\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:31.484225 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.484223    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b43900b-d530-44a1-8e77-d6b412fe2f65-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:31.484405 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.484233    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w257j\" (UniqueName: \"kubernetes.io/projected/7b43900b-d530-44a1-8e77-d6b412fe2f65-kube-api-access-w257j\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:31.484405 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:31.484243    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lrql\" (UniqueName: \"kubernetes.io/projected/1f79e26a-9c6c-4347-8088-2356b2be9445-kube-api-access-6lrql\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:32.227754 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:32.227723    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"
Aug 05 18:18:32.228234 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:32.227758    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v" event={"ID":"384cf89a-b510-478d-9411-85a3f89d7fc7","Type":"ContainerDied","Data":"3aa481ba34be4d280a807897d0430eb85726278413429ca4d0135d1f069269cd"}
Aug 05 18:18:32.228234 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:32.227818    2556 scope.go:117] "RemoveContainer" containerID="4e54b323d4098f8aabf8c3ff9e49e21226582d8fa13fd6310e897afeb43f7bdb"
Aug 05 18:18:32.229218 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:32.229199    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"
Aug 05 18:18:32.229349 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:32.229192    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b" event={"ID":"7b43900b-d530-44a1-8e77-d6b412fe2f65","Type":"ContainerDied","Data":"1fc51846c119dd26e5416cdfd5d9cf5a1c4b667251666923684bbe03c66403d6"}
Aug 05 18:18:32.230649 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:32.230625    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c" event={"ID":"1f79e26a-9c6c-4347-8088-2356b2be9445","Type":"ContainerDied","Data":"613399f2a7e85a27980ee66846c8a82a79ad01011160c6423c07dc4606b1053e"}
Aug 05 18:18:32.230748 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:32.230671    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"
Aug 05 18:18:32.237716 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:32.237689    2556 scope.go:117] "RemoveContainer" containerID="cf3789a3e8c56bcbfdbd0f2291e0e218bd1612f7d79e67e018e816184fa9032a"
Aug 05 18:18:32.246121 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:32.246105    2556 scope.go:117] "RemoveContainer" containerID="3e352b1c6bdafcccdf59e1932740b650eac462e381d3363abffdc86a4f9ac036"
Aug 05 18:18:32.253919 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:32.253895    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"]
Aug 05 18:18:32.257932 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:32.257909    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-3f8e0d-85d6bd595d-c9s9v"]
Aug 05 18:18:32.270115 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:32.270090    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"]
Aug 05 18:18:32.271996 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:32.271972    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3f8e0d-79f79f647f-dfc2b"]
Aug 05 18:18:32.281918 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:32.281899    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"]
Aug 05 18:18:32.285994 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:32.285974    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-3f8e0d-5cddb44b6d-lvn6c"]
Aug 05 18:18:33.096475 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:33.096442    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f79e26a-9c6c-4347-8088-2356b2be9445" path="/var/lib/kubelet/pods/1f79e26a-9c6c-4347-8088-2356b2be9445/volumes"
Aug 05 18:18:33.096804 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:33.096792    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="384cf89a-b510-478d-9411-85a3f89d7fc7" path="/var/lib/kubelet/pods/384cf89a-b510-478d-9411-85a3f89d7fc7/volumes"
Aug 05 18:18:33.097142 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:33.097130    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b43900b-d530-44a1-8e77-d6b412fe2f65" path="/var/lib/kubelet/pods/7b43900b-d530-44a1-8e77-d6b412fe2f65/volumes"
Aug 05 18:18:48.611171 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:48.609824    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"]
Aug 05 18:18:48.611171 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:48.610171    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8" podUID="ced8f1bb-169d-4634-aa79-0664db0f9a23" containerName="maas-api" containerID="cri-o://34891834a91f59aab7d440e790aef47639ac550e3ea7b84f0e53b9390cfbc0f8" gracePeriod=30
Aug 05 18:18:48.662815 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:48.662782    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"]
Aug 05 18:18:48.663057 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:48.663030    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8" podUID="58bbb020-3921-4ed1-b646-eba06286b0de" containerName="payload-processing" containerID="cri-o://f9a32d995cc4adac56905997451ae85589a265e026cf624a371a02b49c2ec009" gracePeriod=30
Aug 05 18:18:48.669256 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:48.669202    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"]
Aug 05 18:18:48.669495 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:48.669442    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp" podUID="565726c0-7400-4c14-a167-b2197ac507b5" containerName="payload-pre-processing" containerID="cri-o://6f24c3fee2e88b2d199bf80fc115d5e0ace9d7931759f7f62f67d9dc619923fb" gracePeriod=30
Aug 05 18:18:48.964847 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:48.964798    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"
Aug 05 18:18:48.967689 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:48.967670    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:18:48.970916 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:48.970894    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"
Aug 05 18:18:49.128967 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.128888    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ced8f1bb-169d-4634-aa79-0664db0f9a23-openshift-service-ca\") pod \"ced8f1bb-169d-4634-aa79-0664db0f9a23\" (UID: \"ced8f1bb-169d-4634-aa79-0664db0f9a23\") "
Aug 05 18:18:49.128967 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.128938    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bjpz\" (UniqueName: \"kubernetes.io/projected/ced8f1bb-169d-4634-aa79-0664db0f9a23-kube-api-access-7bjpz\") pod \"ced8f1bb-169d-4634-aa79-0664db0f9a23\" (UID: \"ced8f1bb-169d-4634-aa79-0664db0f9a23\") "
Aug 05 18:18:49.129227 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.128988    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/565726c0-7400-4c14-a167-b2197ac507b5-plugins-config-volume\") pod \"565726c0-7400-4c14-a167-b2197ac507b5\" (UID: \"565726c0-7400-4c14-a167-b2197ac507b5\") "
Aug 05 18:18:49.129227 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.129008    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2prdj\" (UniqueName: \"kubernetes.io/projected/58bbb020-3921-4ed1-b646-eba06286b0de-kube-api-access-2prdj\") pod \"58bbb020-3921-4ed1-b646-eba06286b0de\" (UID: \"58bbb020-3921-4ed1-b646-eba06286b0de\") "
Aug 05 18:18:49.129227 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.129028    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ced8f1bb-169d-4634-aa79-0664db0f9a23-maas-api-tls\") pod \"ced8f1bb-169d-4634-aa79-0664db0f9a23\" (UID: \"ced8f1bb-169d-4634-aa79-0664db0f9a23\") "
Aug 05 18:18:49.129227 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.129053    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58bbb020-3921-4ed1-b646-eba06286b0de-plugins-config-volume\") pod \"58bbb020-3921-4ed1-b646-eba06286b0de\" (UID: \"58bbb020-3921-4ed1-b646-eba06286b0de\") "
Aug 05 18:18:49.129227 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.129097    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glkgk\" (UniqueName: \"kubernetes.io/projected/565726c0-7400-4c14-a167-b2197ac507b5-kube-api-access-glkgk\") pod \"565726c0-7400-4c14-a167-b2197ac507b5\" (UID: \"565726c0-7400-4c14-a167-b2197ac507b5\") "
Aug 05 18:18:49.129491 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.129419    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/565726c0-7400-4c14-a167-b2197ac507b5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "565726c0-7400-4c14-a167-b2197ac507b5" (UID: "565726c0-7400-4c14-a167-b2197ac507b5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:18:49.129491 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.129428    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ced8f1bb-169d-4634-aa79-0664db0f9a23-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ced8f1bb-169d-4634-aa79-0664db0f9a23" (UID: "ced8f1bb-169d-4634-aa79-0664db0f9a23"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:18:49.129569 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.129507    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58bbb020-3921-4ed1-b646-eba06286b0de-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "58bbb020-3921-4ed1-b646-eba06286b0de" (UID: "58bbb020-3921-4ed1-b646-eba06286b0de"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:18:49.131166 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.131121    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58bbb020-3921-4ed1-b646-eba06286b0de-kube-api-access-2prdj" (OuterVolumeSpecName: "kube-api-access-2prdj") pod "58bbb020-3921-4ed1-b646-eba06286b0de" (UID: "58bbb020-3921-4ed1-b646-eba06286b0de"). InnerVolumeSpecName "kube-api-access-2prdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:18:49.131279 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.131233    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/565726c0-7400-4c14-a167-b2197ac507b5-kube-api-access-glkgk" (OuterVolumeSpecName: "kube-api-access-glkgk") pod "565726c0-7400-4c14-a167-b2197ac507b5" (UID: "565726c0-7400-4c14-a167-b2197ac507b5"). InnerVolumeSpecName "kube-api-access-glkgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:18:49.131569 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.131542    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ced8f1bb-169d-4634-aa79-0664db0f9a23-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ced8f1bb-169d-4634-aa79-0664db0f9a23" (UID: "ced8f1bb-169d-4634-aa79-0664db0f9a23"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:18:49.131569 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.131547    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ced8f1bb-169d-4634-aa79-0664db0f9a23-kube-api-access-7bjpz" (OuterVolumeSpecName: "kube-api-access-7bjpz") pod "ced8f1bb-169d-4634-aa79-0664db0f9a23" (UID: "ced8f1bb-169d-4634-aa79-0664db0f9a23"). InnerVolumeSpecName "kube-api-access-7bjpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:18:49.230304 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.230270    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ced8f1bb-169d-4634-aa79-0664db0f9a23-openshift-service-ca\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:49.230304 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.230301    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bjpz\" (UniqueName: \"kubernetes.io/projected/ced8f1bb-169d-4634-aa79-0664db0f9a23-kube-api-access-7bjpz\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:49.230472 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.230317    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/565726c0-7400-4c14-a167-b2197ac507b5-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:49.230472 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.230330    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2prdj\" (UniqueName: \"kubernetes.io/projected/58bbb020-3921-4ed1-b646-eba06286b0de-kube-api-access-2prdj\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:49.230472 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.230343    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ced8f1bb-169d-4634-aa79-0664db0f9a23-maas-api-tls\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:49.230472 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.230355    2556 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58bbb020-3921-4ed1-b646-eba06286b0de-plugins-config-volume\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:49.230472 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.230369    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glkgk\" (UniqueName: \"kubernetes.io/projected/565726c0-7400-4c14-a167-b2197ac507b5-kube-api-access-glkgk\") on node \"ip-10-0-141-37.ec2.internal\" DevicePath \"\""
Aug 05 18:18:49.314085 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.314052    2556 generic.go:356] "Generic (PLEG): container finished" podID="ced8f1bb-169d-4634-aa79-0664db0f9a23" containerID="34891834a91f59aab7d440e790aef47639ac550e3ea7b84f0e53b9390cfbc0f8" exitCode=0
Aug 05 18:18:49.314229 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.314118    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"
Aug 05 18:18:49.314229 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.314138    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8" event={"ID":"ced8f1bb-169d-4634-aa79-0664db0f9a23","Type":"ContainerDied","Data":"34891834a91f59aab7d440e790aef47639ac550e3ea7b84f0e53b9390cfbc0f8"}
Aug 05 18:18:49.314229 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.314216    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8" event={"ID":"ced8f1bb-169d-4634-aa79-0664db0f9a23","Type":"ContainerDied","Data":"77c115e70b029dbe28c50938d7d610f8391f45ff038f29ec9a290ae4b21b509c"}
Aug 05 18:18:49.314394 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.314250    2556 scope.go:117] "RemoveContainer" containerID="34891834a91f59aab7d440e790aef47639ac550e3ea7b84f0e53b9390cfbc0f8"
Aug 05 18:18:49.315571 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.315544    2556 generic.go:356] "Generic (PLEG): container finished" podID="565726c0-7400-4c14-a167-b2197ac507b5" containerID="6f24c3fee2e88b2d199bf80fc115d5e0ace9d7931759f7f62f67d9dc619923fb" exitCode=0
Aug 05 18:18:49.315703 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.315585    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"
Aug 05 18:18:49.315703 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.315579    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp" event={"ID":"565726c0-7400-4c14-a167-b2197ac507b5","Type":"ContainerDied","Data":"6f24c3fee2e88b2d199bf80fc115d5e0ace9d7931759f7f62f67d9dc619923fb"}
Aug 05 18:18:49.315703 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.315669    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp" event={"ID":"565726c0-7400-4c14-a167-b2197ac507b5","Type":"ContainerDied","Data":"dba259accccf6c351e2f40cdf5bdeb34c93e57bf6990807ab2203faccb0caaae"}
Aug 05 18:18:49.317242 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.317217    2556 generic.go:356] "Generic (PLEG): container finished" podID="58bbb020-3921-4ed1-b646-eba06286b0de" containerID="f9a32d995cc4adac56905997451ae85589a265e026cf624a371a02b49c2ec009" exitCode=0
Aug 05 18:18:49.317371 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.317259    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"
Aug 05 18:18:49.317371 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.317285    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8" event={"ID":"58bbb020-3921-4ed1-b646-eba06286b0de","Type":"ContainerDied","Data":"f9a32d995cc4adac56905997451ae85589a265e026cf624a371a02b49c2ec009"}
Aug 05 18:18:49.317371 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.317318    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8" event={"ID":"58bbb020-3921-4ed1-b646-eba06286b0de","Type":"ContainerDied","Data":"6a58c1c3cea70fa17e8727f0d0c6611ddb51aecd4e9c2297686a5b326fd52b7b"}
Aug 05 18:18:49.323578 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.323561    2556 scope.go:117] "RemoveContainer" containerID="34891834a91f59aab7d440e790aef47639ac550e3ea7b84f0e53b9390cfbc0f8"
Aug 05 18:18:49.323871 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:18:49.323850    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34891834a91f59aab7d440e790aef47639ac550e3ea7b84f0e53b9390cfbc0f8\": container with ID starting with 34891834a91f59aab7d440e790aef47639ac550e3ea7b84f0e53b9390cfbc0f8 not found: ID does not exist" containerID="34891834a91f59aab7d440e790aef47639ac550e3ea7b84f0e53b9390cfbc0f8"
Aug 05 18:18:49.323983 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.323878    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34891834a91f59aab7d440e790aef47639ac550e3ea7b84f0e53b9390cfbc0f8"} err="failed to get container status \"34891834a91f59aab7d440e790aef47639ac550e3ea7b84f0e53b9390cfbc0f8\": rpc error: code = NotFound desc = could not find container \"34891834a91f59aab7d440e790aef47639ac550e3ea7b84f0e53b9390cfbc0f8\": container with ID starting with 34891834a91f59aab7d440e790aef47639ac550e3ea7b84f0e53b9390cfbc0f8 not found: ID does not exist"
Aug 05 18:18:49.323983 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.323899    2556 scope.go:117] "RemoveContainer" containerID="6f24c3fee2e88b2d199bf80fc115d5e0ace9d7931759f7f62f67d9dc619923fb"
Aug 05 18:18:49.332691 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.332673    2556 scope.go:117] "RemoveContainer" containerID="6f24c3fee2e88b2d199bf80fc115d5e0ace9d7931759f7f62f67d9dc619923fb"
Aug 05 18:18:49.332957 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:18:49.332939    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f24c3fee2e88b2d199bf80fc115d5e0ace9d7931759f7f62f67d9dc619923fb\": container with ID starting with 6f24c3fee2e88b2d199bf80fc115d5e0ace9d7931759f7f62f67d9dc619923fb not found: ID does not exist" containerID="6f24c3fee2e88b2d199bf80fc115d5e0ace9d7931759f7f62f67d9dc619923fb"
Aug 05 18:18:49.333002 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.332963    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f24c3fee2e88b2d199bf80fc115d5e0ace9d7931759f7f62f67d9dc619923fb"} err="failed to get container status \"6f24c3fee2e88b2d199bf80fc115d5e0ace9d7931759f7f62f67d9dc619923fb\": rpc error: code = NotFound desc = could not find container \"6f24c3fee2e88b2d199bf80fc115d5e0ace9d7931759f7f62f67d9dc619923fb\": container with ID starting with 6f24c3fee2e88b2d199bf80fc115d5e0ace9d7931759f7f62f67d9dc619923fb not found: ID does not exist"
Aug 05 18:18:49.333002 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.332978    2556 scope.go:117] "RemoveContainer" containerID="f9a32d995cc4adac56905997451ae85589a265e026cf624a371a02b49c2ec009"
Aug 05 18:18:49.340628 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.340610    2556 scope.go:117] "RemoveContainer" containerID="f9a32d995cc4adac56905997451ae85589a265e026cf624a371a02b49c2ec009"
Aug 05 18:18:49.340865 ip-10-0-141-37 kubenswrapper[2556]: E0805 18:18:49.340848    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9a32d995cc4adac56905997451ae85589a265e026cf624a371a02b49c2ec009\": container with ID starting with f9a32d995cc4adac56905997451ae85589a265e026cf624a371a02b49c2ec009 not found: ID does not exist" containerID="f9a32d995cc4adac56905997451ae85589a265e026cf624a371a02b49c2ec009"
Aug 05 18:18:49.340933 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.340869    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9a32d995cc4adac56905997451ae85589a265e026cf624a371a02b49c2ec009"} err="failed to get container status \"f9a32d995cc4adac56905997451ae85589a265e026cf624a371a02b49c2ec009\": rpc error: code = NotFound desc = could not find container \"f9a32d995cc4adac56905997451ae85589a265e026cf624a371a02b49c2ec009\": container with ID starting with f9a32d995cc4adac56905997451ae85589a265e026cf624a371a02b49c2ec009 not found: ID does not exist"
Aug 05 18:18:49.356287 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.356262    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"]
Aug 05 18:18:49.371540 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.371506    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-30613e-5b86c75688-vmwhp"]
Aug 05 18:18:49.389434 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.389388    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"]
Aug 05 18:18:49.403078 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.403057    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-30613e-77f775b9dd-g45h8"]
Aug 05 18:18:49.415084 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.415062    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"]
Aug 05 18:18:49.418777 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:49.418756    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-30613e-858c669bd6-fkjj8"]
Aug 05 18:18:51.097299 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:51.097270    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="565726c0-7400-4c14-a167-b2197ac507b5" path="/var/lib/kubelet/pods/565726c0-7400-4c14-a167-b2197ac507b5/volumes"
Aug 05 18:18:51.097650 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:51.097611    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58bbb020-3921-4ed1-b646-eba06286b0de" path="/var/lib/kubelet/pods/58bbb020-3921-4ed1-b646-eba06286b0de/volumes"
Aug 05 18:18:51.097950 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:18:51.097937    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ced8f1bb-169d-4634-aa79-0664db0f9a23" path="/var/lib/kubelet/pods/ced8f1bb-169d-4634-aa79-0664db0f9a23/volumes"
Aug 05 18:19:01.638074 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:01.638041    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-48244_a86647ff-9a52-4ae9-85fa-421e108a9805/manager/0.log"
Aug 05 18:19:01.763620 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:01.763588    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hd9qj_93bced0f-67fa-4d9e-b335-1970769ca3e5/manager/0.log"
Aug 05 18:19:01.890313 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:01.890226    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c55ff665-tctsm_daa1e6b3-6cd2-483a-8530-536be6b95ae3/manager/0.log"
Aug 05 18:19:02.018541 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:02.018513    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-h69xq_40aca00f-bca1-4a21-a4c3-b18adf59745e/server/0.log"
Aug 05 18:19:02.497551 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:02.497523    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58c579b484-dzh4p_cbdd8ac3-41d3-4941-a2ec-a764a2640b37/copy-manifests/0.log"
Aug 05 18:19:02.514395 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:02.514368    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58c579b484-dzh4p_cbdd8ac3-41d3-4941-a2ec-a764a2640b37/manager/0.log"
Aug 05 18:19:02.871905 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:02.871806    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6475fbf58c-jt7dw_f8c8e850-d79f-4c86-917e-0b8bf074e986/maas-api/0.log"
Aug 05 18:19:03.333589 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:03.333560    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-fjh8r_c72eab45-b832-4766-a2ff-d2562066b00a/postgres/0.log"
Aug 05 18:19:04.270973 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:04.270945    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-rr8ht_578f5265-ae41-4488-aff0-b03cf4c05cbe/manager/0.log"
Aug 05 18:19:04.633115 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:04.633008    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-8v9h2_73cf224c-963a-4dd4-bc75-102512457040/manager/0.log"
Aug 05 18:19:04.863122 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:04.863092    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-5mqs5_af372020-540d-40d5-80c8-900b3b99b3a4/manager/0.log"
Aug 05 18:19:05.450999 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:05.450970    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c99fc649-hq2hm_2686161c-63d4-4b55-9f0f-0583e8ce7f56/kube-auth-proxy/0.log"
Aug 05 18:19:05.813104 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:05.813069    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-z9rmx_0244cf89-0bf2-48ba-bb1c-8e9980a77ff6/payload-pre-processing/0.log"
Aug 05 18:19:05.939184 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:05.939135    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w7ssw_522e9cc8-a0a6-45f6-896a-1b893541b01e/payload-processing/0.log"
Aug 05 18:19:06.518728 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:06.518696    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-g2x5t_de8ba1b9-6b73-4600-8328-8df55c054521/main/0.log"
Aug 05 18:19:06.862357 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:06.862266    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-ckv2c_1c45a533-b352-4c58-b99a-2c91cb3bb618/main/0.log"
Aug 05 18:19:06.970386 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:06.970343    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-nbs4z_996a6bf5-1fab-47f7-a31d-50ab1c2639fb/main/0.log"
Aug 05 18:19:15.625798 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:15.625763    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-t2vbt_46b91743-4ee2-4ac1-9779-5b85586520ec/global-pull-secret-syncer/0.log"
Aug 05 18:19:15.806581 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:15.806546    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-q4cgr_6489ed96-823a-4ff5-bab4-0efabcb3584b/konnectivity-agent/0.log"
Aug 05 18:19:15.849770 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:15.849729    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-37.ec2.internal_13b14f86a6dd04037249bf3e254a8bd2/haproxy/0.log"
Aug 05 18:19:21.166936 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:21.166901    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-rr8ht_578f5265-ae41-4488-aff0-b03cf4c05cbe/manager/0.log"
Aug 05 18:19:21.365955 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:21.365920    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-8v9h2_73cf224c-963a-4dd4-bc75-102512457040/manager/0.log"
Aug 05 18:19:21.525298 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:21.525267    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-5mqs5_af372020-540d-40d5-80c8-900b3b99b3a4/manager/0.log"
Aug 05 18:19:23.256050 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:23.256013    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5fslc_ac88833f-ac4d-4f2c-a09c-e4259988f433/node-exporter/0.log"
Aug 05 18:19:23.280980 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:23.280959    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5fslc_ac88833f-ac4d-4f2c-a09c-e4259988f433/kube-rbac-proxy/0.log"
Aug 05 18:19:23.303659 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:23.303639    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5fslc_ac88833f-ac4d-4f2c-a09c-e4259988f433/init-textfile/0.log"
Aug 05 18:19:25.431330 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:25.431297    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-7rkk2_1dd38456-9704-41f6-bb50-f1dab34238ca/networking-console-plugin/0.log"
Aug 05 18:19:26.227338 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227302    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"]
Aug 05 18:19:26.227725 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227707    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ced8f1bb-169d-4634-aa79-0664db0f9a23" containerName="maas-api"
Aug 05 18:19:26.227820 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227728    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ced8f1bb-169d-4634-aa79-0664db0f9a23" containerName="maas-api"
Aug 05 18:19:26.227820 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227744    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d0bd32d-3d66-4dbd-be05-66d52a429d7d" containerName="payload-pre-processing"
Aug 05 18:19:26.227820 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227752    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d0bd32d-3d66-4dbd-be05-66d52a429d7d" containerName="payload-pre-processing"
Aug 05 18:19:26.227820 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227764    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b43900b-d530-44a1-8e77-d6b412fe2f65" containerName="maas-api"
Aug 05 18:19:26.227820 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227773    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b43900b-d530-44a1-8e77-d6b412fe2f65" containerName="maas-api"
Aug 05 18:19:26.227820 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227785    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f79e26a-9c6c-4347-8088-2356b2be9445" containerName="payload-pre-processing"
Aug 05 18:19:26.227820 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227793    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f79e26a-9c6c-4347-8088-2356b2be9445" containerName="payload-pre-processing"
Aug 05 18:19:26.227820 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227803    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4dde33d-914d-4c61-a3b7-78897859fe6b" containerName="payload-processing"
Aug 05 18:19:26.227820 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227811    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4dde33d-914d-4c61-a3b7-78897859fe6b" containerName="payload-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227825    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e" containerName="payload-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227833    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e" containerName="payload-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227844    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58bbb020-3921-4ed1-b646-eba06286b0de" containerName="payload-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227855    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="58bbb020-3921-4ed1-b646-eba06286b0de" containerName="payload-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227864    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0289f52e-9c7f-401a-8b3f-4643af4ed035" containerName="payload-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227872    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0289f52e-9c7f-401a-8b3f-4643af4ed035" containerName="payload-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227890    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b64207e7-5f7f-4e0d-9528-930eb868ce0f" containerName="payload-pre-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227899    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b64207e7-5f7f-4e0d-9528-930eb868ce0f" containerName="payload-pre-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227916    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="384cf89a-b510-478d-9411-85a3f89d7fc7" containerName="payload-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227926    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="384cf89a-b510-478d-9411-85a3f89d7fc7" containerName="payload-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227934    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c37e21f9-3cad-4088-8537-d336b7205930" containerName="maas-api"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227942    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c37e21f9-3cad-4088-8537-d336b7205930" containerName="maas-api"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227958    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="565726c0-7400-4c14-a167-b2197ac507b5" containerName="payload-pre-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.227967    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="565726c0-7400-4c14-a167-b2197ac507b5" containerName="payload-pre-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.228052    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3d53188-1f55-4fb5-ab85-5a51a4b1ad2e" containerName="payload-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.228063    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4dde33d-914d-4c61-a3b7-78897859fe6b" containerName="payload-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.228073    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d0bd32d-3d66-4dbd-be05-66d52a429d7d" containerName="payload-pre-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.228083    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ced8f1bb-169d-4634-aa79-0664db0f9a23" containerName="maas-api"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.228092    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b43900b-d530-44a1-8e77-d6b412fe2f65" containerName="maas-api"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.228101    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="565726c0-7400-4c14-a167-b2197ac507b5" containerName="payload-pre-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.228110    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="384cf89a-b510-478d-9411-85a3f89d7fc7" containerName="payload-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.228119    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="58bbb020-3921-4ed1-b646-eba06286b0de" containerName="payload-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.228128    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f79e26a-9c6c-4347-8088-2356b2be9445" containerName="payload-pre-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.228136    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c37e21f9-3cad-4088-8537-d336b7205930" containerName="maas-api"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.228145    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b64207e7-5f7f-4e0d-9528-930eb868ce0f" containerName="payload-pre-processing"
Aug 05 18:19:26.228300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.228184    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0289f52e-9c7f-401a-8b3f-4643af4ed035" containerName="payload-processing"
Aug 05 18:19:26.231035 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.231017    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.233811 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.233790    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-f9829\"/\"default-dockercfg-6655g\""
Aug 05 18:19:26.233811 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.233806    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f9829\"/\"kube-root-ca.crt\""
Aug 05 18:19:26.234707 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.234691    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f9829\"/\"openshift-service-ca.crt\""
Aug 05 18:19:26.239123 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.239098    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"]
Aug 05 18:19:26.302218 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.302189    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e9a5008-4a1a-490a-97e2-60f1a2d53200-proc\") pod \"perf-node-gather-daemonset-k4kjm\" (UID: \"8e9a5008-4a1a-490a-97e2-60f1a2d53200\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.302352 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.302224    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e9a5008-4a1a-490a-97e2-60f1a2d53200-sys\") pod \"perf-node-gather-daemonset-k4kjm\" (UID: \"8e9a5008-4a1a-490a-97e2-60f1a2d53200\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.302352 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.302246    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e9a5008-4a1a-490a-97e2-60f1a2d53200-lib-modules\") pod \"perf-node-gather-daemonset-k4kjm\" (UID: \"8e9a5008-4a1a-490a-97e2-60f1a2d53200\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.302352 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.302292    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vznsm\" (UniqueName: \"kubernetes.io/projected/8e9a5008-4a1a-490a-97e2-60f1a2d53200-kube-api-access-vznsm\") pod \"perf-node-gather-daemonset-k4kjm\" (UID: \"8e9a5008-4a1a-490a-97e2-60f1a2d53200\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.302491 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.302361    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e9a5008-4a1a-490a-97e2-60f1a2d53200-podres\") pod \"perf-node-gather-daemonset-k4kjm\" (UID: \"8e9a5008-4a1a-490a-97e2-60f1a2d53200\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.403677 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.403655    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e9a5008-4a1a-490a-97e2-60f1a2d53200-sys\") pod \"perf-node-gather-daemonset-k4kjm\" (UID: \"8e9a5008-4a1a-490a-97e2-60f1a2d53200\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.403780 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.403683    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e9a5008-4a1a-490a-97e2-60f1a2d53200-lib-modules\") pod \"perf-node-gather-daemonset-k4kjm\" (UID: \"8e9a5008-4a1a-490a-97e2-60f1a2d53200\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.403780 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.403702    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vznsm\" (UniqueName: \"kubernetes.io/projected/8e9a5008-4a1a-490a-97e2-60f1a2d53200-kube-api-access-vznsm\") pod \"perf-node-gather-daemonset-k4kjm\" (UID: \"8e9a5008-4a1a-490a-97e2-60f1a2d53200\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.403780 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.403738    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e9a5008-4a1a-490a-97e2-60f1a2d53200-podres\") pod \"perf-node-gather-daemonset-k4kjm\" (UID: \"8e9a5008-4a1a-490a-97e2-60f1a2d53200\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.403780 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.403772    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e9a5008-4a1a-490a-97e2-60f1a2d53200-sys\") pod \"perf-node-gather-daemonset-k4kjm\" (UID: \"8e9a5008-4a1a-490a-97e2-60f1a2d53200\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.403955 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.403788    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e9a5008-4a1a-490a-97e2-60f1a2d53200-proc\") pod \"perf-node-gather-daemonset-k4kjm\" (UID: \"8e9a5008-4a1a-490a-97e2-60f1a2d53200\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.403955 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.403843    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8e9a5008-4a1a-490a-97e2-60f1a2d53200-proc\") pod \"perf-node-gather-daemonset-k4kjm\" (UID: \"8e9a5008-4a1a-490a-97e2-60f1a2d53200\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.403955 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.403850    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e9a5008-4a1a-490a-97e2-60f1a2d53200-lib-modules\") pod \"perf-node-gather-daemonset-k4kjm\" (UID: \"8e9a5008-4a1a-490a-97e2-60f1a2d53200\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.403955 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.403881    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8e9a5008-4a1a-490a-97e2-60f1a2d53200-podres\") pod \"perf-node-gather-daemonset-k4kjm\" (UID: \"8e9a5008-4a1a-490a-97e2-60f1a2d53200\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.412343 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.412322    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vznsm\" (UniqueName: \"kubernetes.io/projected/8e9a5008-4a1a-490a-97e2-60f1a2d53200-kube-api-access-vznsm\") pod \"perf-node-gather-daemonset-k4kjm\" (UID: \"8e9a5008-4a1a-490a-97e2-60f1a2d53200\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.540969 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.540901    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:26.671567 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:26.671538    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"]
Aug 05 18:19:26.673105 ip-10-0-141-37 kubenswrapper[2556]: W0805 18:19:26.673072    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8e9a5008_4a1a_490a_97e2_60f1a2d53200.slice/crio-44ba330f70dd19cf6f399571745a0c6bda88e22cb9669ed6a7c97dfb09935d5c WatchSource:0}: Error finding container 44ba330f70dd19cf6f399571745a0c6bda88e22cb9669ed6a7c97dfb09935d5c: Status 404 returned error can't find the container with id 44ba330f70dd19cf6f399571745a0c6bda88e22cb9669ed6a7c97dfb09935d5c
Aug 05 18:19:27.495961 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:27.495924    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm" event={"ID":"8e9a5008-4a1a-490a-97e2-60f1a2d53200","Type":"ContainerStarted","Data":"f5e80636b2b4b064d74d320cbc2b7c635fe609e6cc6d50f9576bc59f49bf3b5b"}
Aug 05 18:19:27.496115 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:27.496019    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:27.496115 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:27.496043    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm" event={"ID":"8e9a5008-4a1a-490a-97e2-60f1a2d53200","Type":"ContainerStarted","Data":"44ba330f70dd19cf6f399571745a0c6bda88e22cb9669ed6a7c97dfb09935d5c"}
Aug 05 18:19:27.513260 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:27.513215    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm" podStartSLOduration=1.51320214 podStartE2EDuration="1.51320214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 18:19:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:19:27.512242155 +0000 UTC m=+3580.930556935" watchObservedRunningTime="2026-08-05 18:19:27.51320214 +0000 UTC m=+3580.931516897"
Aug 05 18:19:28.345132 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:28.345102    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6dqwh_07ff7d67-32e8-4c41-846d-ba8fa6400a52/dns/0.log"
Aug 05 18:19:28.365548 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:28.365526    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6dqwh_07ff7d67-32e8-4c41-846d-ba8fa6400a52/kube-rbac-proxy/0.log"
Aug 05 18:19:28.479130 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:28.479103    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5z6td_e8d949ff-bbbc-4247-87f0-d9f98e7ec389/dns-node-resolver/0.log"
Aug 05 18:19:29.127536 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:29.127495    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-l2prc_a264b228-b7f6-4977-b858-820fc604b53c/node-ca/0.log"
Aug 05 18:19:30.254076 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:30.254049    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c99fc649-hq2hm_2686161c-63d4-4b55-9f0f-0583e8ce7f56/kube-auth-proxy/0.log"
Aug 05 18:19:30.355435 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:30.355395    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-z9rmx_0244cf89-0bf2-48ba-bb1c-8e9980a77ff6/payload-pre-processing/0.log"
Aug 05 18:19:30.395182 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:30.395138    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w7ssw_522e9cc8-a0a6-45f6-896a-1b893541b01e/payload-processing/0.log"
Aug 05 18:19:31.303119 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:31.303088    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-svkcg_a5c8f0bd-ce04-46b6-8a24-5fefe12683d3/serve-healthcheck-canary/0.log"
Aug 05 18:19:31.799945 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:31.799914    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d4wnf_0d9598a7-fbf8-4772-859a-6765db90a1b5/kube-rbac-proxy/0.log"
Aug 05 18:19:31.822575 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:31.822549    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d4wnf_0d9598a7-fbf8-4772-859a-6765db90a1b5/exporter/0.log"
Aug 05 18:19:31.844206 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:31.844185    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d4wnf_0d9598a7-fbf8-4772-859a-6765db90a1b5/extractor/0.log"
Aug 05 18:19:33.512002 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:33.511975    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-f9829/perf-node-gather-daemonset-k4kjm"
Aug 05 18:19:34.048593 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:34.048552    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-48244_a86647ff-9a52-4ae9-85fa-421e108a9805/manager/0.log"
Aug 05 18:19:34.474885 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:34.474846    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hd9qj_93bced0f-67fa-4d9e-b335-1970769ca3e5/manager/0.log"
Aug 05 18:19:34.769823 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:34.769791    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c55ff665-tctsm_daa1e6b3-6cd2-483a-8530-536be6b95ae3/manager/0.log"
Aug 05 18:19:34.789300 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:34.789278    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-h69xq_40aca00f-bca1-4a21-a4c3-b18adf59745e/server/0.log"
Aug 05 18:19:35.132103 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:35.132025    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58c579b484-dzh4p_cbdd8ac3-41d3-4941-a2ec-a764a2640b37/manager/0.log"
Aug 05 18:19:35.148392 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:35.148368    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58c579b484-dzh4p_cbdd8ac3-41d3-4941-a2ec-a764a2640b37/copy-manifests/0.log"
Aug 05 18:19:36.872953 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:36.872921    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c99fc649-hq2hm_2686161c-63d4-4b55-9f0f-0583e8ce7f56/kube-auth-proxy/0.log"
Aug 05 18:19:36.952099 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:36.952075    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-z9rmx_0244cf89-0bf2-48ba-bb1c-8e9980a77ff6/payload-pre-processing/0.log"
Aug 05 18:19:36.982832 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:36.982804    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w7ssw_522e9cc8-a0a6-45f6-896a-1b893541b01e/payload-processing/0.log"
Aug 05 18:19:38.585397 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:38.585362    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c99fc649-hq2hm_2686161c-63d4-4b55-9f0f-0583e8ce7f56/kube-auth-proxy/0.log"
Aug 05 18:19:38.660774 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:38.660741    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-z9rmx_0244cf89-0bf2-48ba-bb1c-8e9980a77ff6/payload-pre-processing/0.log"
Aug 05 18:19:38.694084 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:38.694047    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-w7ssw_522e9cc8-a0a6-45f6-896a-1b893541b01e/payload-processing/0.log"
Aug 05 18:19:41.608210 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:41.608181    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-68kpx_1395403f-dd78-419d-b2f8-b0fc17073f9d/migrator/0.log"
Aug 05 18:19:41.630993 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:41.630969    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-68kpx_1395403f-dd78-419d-b2f8-b0fc17073f9d/graceful-termination/0.log"
Aug 05 18:19:43.025818 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:43.025786    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-74dt8_acc396cc-8191-48c6-8ba3-c6e0fd95f0fa/kube-multus/0.log"
Aug 05 18:19:43.436760 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:43.436684    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzlln_4d033317-735c-4582-b54f-f63c0e0663fb/kube-multus-additional-cni-plugins/0.log"
Aug 05 18:19:43.480610 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:43.480583    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzlln_4d033317-735c-4582-b54f-f63c0e0663fb/egress-router-binary-copy/0.log"
Aug 05 18:19:43.502102 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:43.502079    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzlln_4d033317-735c-4582-b54f-f63c0e0663fb/cni-plugins/0.log"
Aug 05 18:19:43.522941 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:43.522921    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzlln_4d033317-735c-4582-b54f-f63c0e0663fb/bond-cni-plugin/0.log"
Aug 05 18:19:43.545826 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:43.545805    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzlln_4d033317-735c-4582-b54f-f63c0e0663fb/routeoverride-cni/0.log"
Aug 05 18:19:43.568795 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:43.568777    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzlln_4d033317-735c-4582-b54f-f63c0e0663fb/whereabouts-cni-bincopy/0.log"
Aug 05 18:19:43.590083 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:43.590062    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzlln_4d033317-735c-4582-b54f-f63c0e0663fb/whereabouts-cni/0.log"
Aug 05 18:19:43.743023 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:43.743001    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8fkfq_bfd52bcf-e224-4a4c-8fa2-13c94f985e60/network-metrics-daemon/0.log"
Aug 05 18:19:43.761466 ip-10-0-141-37 kubenswrapper[2556]: I0805 18:19:43.761445    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8fkfq_bfd52bcf-e224-4a4c-8fa2-13c94f985e60/kube-rbac-proxy/0.log"